-- Logs begin at Mon 2022-06-20 06:18:43 UTC, end at Mon 2022-06-20 07:54:06 UTC. -- Jun 20 06:53:01.913015 dv-cinder-778193-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 20 06:53:01.923937 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 20 06:53:01 2022] *** Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: nodename: dv-cinder-778193-28-iscsi Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: machine: x86_64 Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: clock source: unix Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: pcre jit disabled Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: detected number of CPU cores: 4 Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: current working directory: / Jun 20 06:53:01.925994 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: detected binary path: /usr/bin/uwsgi-core Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: your processes number limit is 47680 Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: your memory page size is 4096 bytes Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: detected max file descriptor number: 2048 Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: lock engine: pthread robust mutexes Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: thunder lock: enabled Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 20 06:53:01.926623 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 20 06:53:01.969841 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: Python main interpreter initialized at 0x558f6c62c9c0 Jun 20 06:53:01.969841 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: python threads support enabled Jun 20 06:53:01.969841 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: your server socket listen backlog is limited to 100 connections Jun 20 06:53:01.969841 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: your mercy for graceful operations on workers is 90 seconds Jun 20 06:53:01.970131 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: mapped 403077 bytes (393 KB) for 2 cores Jun 20 06:53:01.970204 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: *** Operational MODE: preforking *** Jun 20 06:53:01.970452 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: *** uWSGI is running in multiple interpreter mode *** Jun 20 06:53:01.970452 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: spawned uWSGI master process (pid: 80125) Jun 20 06:53:01.970625 dv-cinder-778193-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 20 06:53:01.971530 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: spawned uWSGI worker 1 (pid: 80136, cores: 1) Jun 20 06:53:01.971530 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: spawned uWSGI worker 2 (pid: 80137, cores: 1) Jun 20 06:53:01.971683 dv-cinder-778193-28-iscsi devstack@keystone.service[80125]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 20 06:53:07.398589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 06:53:07.398589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: from cryptography.utils import int_from_bytes Jun 20 06:53:07.399390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 06:53:07.399390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: from cryptography.utils import int_from_bytes Jun 20 06:53:07.399390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 06:53:07.399390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: from cryptography.utils import int_from_bytes Jun 20 06:53:07.399876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 06:53:07.399876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: from cryptography.utils import int_from_bytes Jun 20 06:53:07.448166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 06:53:07.448343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 06:53:07.448544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] command line args: [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 06:53:07.448635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 06:53:07.448746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 06:53:07.448823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 06:53:07.448941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] command line args: [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 06:53:07.449019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ================================================================================ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 06:53:07.449212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 06:53:07.449295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] admin_token = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.449391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ================================================================================ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 06:53:07.449467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_dir = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.449638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.449721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] admin_token = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.449825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] config_source = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.449907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_dir = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.450006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] debug = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.450091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.450205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] config_source = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] debug = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.452876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_config_append = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_config_append = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_dir = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_dir = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.453869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.454986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.455944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] publish_errors = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] publish_errors = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.456915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] standard_threads = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] standard_threads = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.457935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_journal = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_journal = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_json = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.458909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_json = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_stderr = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_stderr = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] use_syslog = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] use_syslog = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 06:53:07.459729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.459861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.459976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.460939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.461943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.462914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.463908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.464940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.465864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.466968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.467895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.468898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.469950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.470967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.471991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.472094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.472302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.472415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.473970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.474899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.475929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.476989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.477907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.478943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.479989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.480898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.481944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.482982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.483905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.external = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.external = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.password = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.484994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.password = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] auth.token = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] auth.token = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.485967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.486910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.487957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.488076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.488899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.489958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.490997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.491916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.492973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.493900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.494935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.495854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.496895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.497927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.498881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.499914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.500979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.501896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.502941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.503902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.504960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.505892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.506924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.507967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.508956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.509960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.510991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.511898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.512995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.513918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.514959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.515982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.516999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.517929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.518976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.519930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.520887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.521930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.522971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.523892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.524913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.525953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.526991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.527909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.528884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.529932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.530886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.531922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.532984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.533907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.534021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.534135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.534257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.534257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80137) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 06:53:07.534473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 06:53:07.534473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80136) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 06:53:07.563793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.563974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.564077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.564169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.564282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.564368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.572828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.573035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.601532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.601757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.601997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.602228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.742855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.744498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.791952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.792207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.792580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.792815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.855940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.856967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.857092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.857181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.857337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.881652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.881896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.882992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.883075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.883247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.883462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.894054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.894236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.907980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80136) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.908772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80137) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 06:53:07.926419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.926601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.926694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 06:53:07.926808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 06:53:07.926948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 06:53:07.927335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 06:53:07.927418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.927937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.928126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.928212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.928383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.938978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.939886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.940132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.948189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.948458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.951483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.951724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.951808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.951996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.952916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.953938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.954120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.954199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.954377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.981777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.981874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.983346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.983432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.983623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.983706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.988971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.989153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.989330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.989547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.989654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.989827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.996941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.997217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:07.997301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:07.997543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.001681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.001961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.002238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.002867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.003655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.004412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.023507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.023773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.023925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.024232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.028442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.028883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.032039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.032461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.032543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.032911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.037527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.037944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.041147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.041566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.044860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.045984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.046068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.046255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.046339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.046587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.065807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.066085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.066880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.067146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.068219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.068503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.068770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.069034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.074980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.076703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.076969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.077091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.078687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.078962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.080217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.080512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.080806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.081169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.081450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.082851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.083147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.083446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.083743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.084075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.095151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.095426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.095708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.095985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.097367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.097573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.097670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.097859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.097941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.098148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.098231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.099767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.100060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.100375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.116089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.118269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.119373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.121586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.122670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.124406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.124916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.126028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.126301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.126576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.126657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.128357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.128634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.129451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.134400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.135998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.136812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.138423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.139225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.139510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.139898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.140204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.141689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.141977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.142361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.142636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.150077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.150359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.150635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.152597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.152880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.153157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.158466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80137) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.158738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.159010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.159274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.159538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.159925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.160293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.160572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.160841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80136) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 06:53:08.161171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.161984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.162160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80137) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.162527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.162817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.163087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.163358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.163634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.163905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.164195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.164473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.164743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80136) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 06:53:08.200460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558f6c62c9c0 pid: 80137 (default app) Jun 20 06:53:08.202888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x558f6c62c9c0 pid: 80136 (default app) Jun 20 06:53:08.203831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197d9b1-b64d-49d8-b291-5c7322c9e1cc None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:08.204152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197d9b1-b64d-49d8-b291-5c7322c9e1cc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:08.204433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197d9b1-b64d-49d8-b291-5c7322c9e1cc None None] PATH_INFO: `/v3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:08.209166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Mon Jun 20 06:53:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:15.828083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:15.828655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:15.828655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:15.833572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:15.839321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71049-626e-4102-b636-099d6d96286e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:15.839761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71049-626e-4102-b636-099d6d96286e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:15.839889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c71049-626e-4102-b636-099d6d96286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:15.909164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: INFO dbcounter [None req-b6c71049-626e-4102-b636-099d6d96286e None None] Registered counter for database keystone Jun 20 06:53:15.930402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG oslo_db.sqlalchemy.engines [None req-b6c71049-626e-4102-b636-099d6d96286e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80137) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 06:53:15.936489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writer thread running {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 06:53:16.124640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG passlib.handlers.bcrypt [None req-b6c71049-626e-4102-b636-099d6d96286e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80137) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 06:53:16.125054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG passlib.handlers.bcrypt [None req-b6c71049-626e-4102-b636-099d6d96286e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80137) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 06:53:16.179735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b6c71049-626e-4102-b636-099d6d96286e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:16.181329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b6c71049-626e-4102-b636-099d6d96286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:16.300682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 20 06:53:16.435301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:16.505494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: INFO dbcounter [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] Registered counter for database keystone Jun 20 06:53:16.524792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG oslo_db.sqlalchemy.engines [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80136) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 06:53:16.530934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writer thread running {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 06:53:16.653948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48f911-6d43-4b37-91a3-da2ac58b6ba9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:16.658970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:16.658970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:16.659230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:16.659431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:16.668525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:16.720907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 06:53:16.721257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af4dd1f-7c34-4de1-85fd-0fe6eb412406 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:16.731288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Mon Jun 20 06:53:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 06:53:16.737175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71049-626e-4102-b636-099d6d96286e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:16.754891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c71049-626e-4102-b636-099d6d96286e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:16.760222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:16.761350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:16.761645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:16.761929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] PATH_INFO: `/v3/projects/649b350e7f364263b37fbd34d0f07b2b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:16.766344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] RBAC: Authorizing `identity:get_project(project_id=649b350e7f364263b37fbd34d0f07b2b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:16.820318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING keystone.common.rbac_enforcer.enforcer [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 06:53:16.820664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8906-6af4-4483-9ab0-93c2e49207d7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:16.825336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Mon Jun 20 06:53:16 2022] GET /identity/v3/projects/649b350e7f364263b37fbd34d0f07b2b => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:18.853217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943576c-b4dd-4c0e-8830-07c220ca927b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:18.853712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943576c-b4dd-4c0e-8830-07c220ca927b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:18.853815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943576c-b4dd-4c0e-8830-07c220ca927b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:18.857869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:18.863728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:18.864141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:18.864316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:18.883306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:18.884880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:18.892980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 20 06:53:18.914789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943576c-b4dd-4c0e-8830-07c220ca927b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:18.927123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943576c-b4dd-4c0e-8830-07c220ca927b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:18.931198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:18.931822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:18.932123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:18.932415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:18.940037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:18.942401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aade1c-9602-4ef1-ba2d-cd7cb032106b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:18.950983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Mon Jun 20 06:53:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:53:18.956318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:18.965256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31408547-6bf6-47fc-8bc0-9a5f3b956f07 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:18.969575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:18.970427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:18.970721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:18.971003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:18.972421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:18.974244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f57a09-b62e-49c1-b556-fbeaf7aa5760 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:18.975800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Mon Jun 20 06:53:18 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:21.075274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94036-b3fd-4283-9e98-9ef17335d0b7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.075890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94036-b3fd-4283-9e98-9ef17335d0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.076144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94036-b3fd-4283-9e98-9ef17335d0b7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.077130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01772c-f1f3-451c-a4f5-eba883b5f617 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.077496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01772c-f1f3-451c-a4f5-eba883b5f617 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.077655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:21.077786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01772c-f1f3-451c-a4f5-eba883b5f617 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.079027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:21.082960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.083224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.083507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.084592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.084882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.085161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.101309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:21.102499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:21.122423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG passlib.handlers.bcrypt [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80136) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 06:53:21.122849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG passlib.handlers.bcrypt [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80136) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 06:53:21.168559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:21.170152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:21.187304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 7/12] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:21.210946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.233743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.270379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 6/13] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:21.292589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.313557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7e9cb-ff1d-41fe-a195-f01c9397a450 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.347644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.348415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.348708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.348990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.359913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:21.371555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:21.372530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:21.373350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 8/14] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:21.379174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.390828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59373d-5799-439a-9898-ee626ae9de44 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.393157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.428631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.429527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.429825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.430108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.464311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.483656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085b7dd-7d8a-4ee8-98d8-c4a911579f8f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.500343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.501887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.517943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.518610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.519155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.519434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.520213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:21.520522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.521908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.533471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 9/15] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:21.539464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.599194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 7/16] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:53:21.604651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.607304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.640266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2e8c4-4ea7-4cf4-b69a-78ea11a9f914 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.675273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.676162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.676456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.676734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.677949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.679417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.681072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 10/17] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:21.686319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.700455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.706458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7b21d-7206-4df6-aed8-978e108b956e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.741799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.742602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.742892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.743174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.762455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.763926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:53:21.784786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:21.785666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:21.786482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:53:21.786482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:53:21.786482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:53:21.786482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:21.786708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b26a9-1134-4a87-b328-8790aeccb10c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.786777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 8/18] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:21.791689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.820549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.821338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.821624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.821901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.823076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.824552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.826141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 11/19] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:21.832172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:21.848329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.891703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50f262-6c13-47de-9d67-03cffe3ab20f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.911909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:21.926014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.926673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.926963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.927245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.927974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:21.928313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.929730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f0072-23e5-4583-b9f2-473c20b6b6ae None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.931311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221d3ef-00b5-4336-b5c9-8931776f4d7c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:21.946809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 9/20] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:21.965717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:21.966400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:21.966691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:21.966971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:21.967838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.domain.id=default, user.id=1ecb0c12822b4be89d3c7e17c8adfe7e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:21.968185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:21.969687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76b6d-c7a4-4c92-970b-5cf5cc812d74 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:21.978070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 12/21] 10.222.0.23 () {56 vars in 1322 bytes} [Mon Jun 20 06:53:21 2022] GET /identity/v3/role_assignments?user.id=1ecb0c12822b4be89d3c7e17c8adfe7e&scope.domain.id=default&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:24.363489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8fa9-5583-4f60-aba1-a206ad074a49 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.363999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8fa9-5583-4f60-aba1-a206ad074a49 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.364074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc8fa9-5583-4f60-aba1-a206ad074a49 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.365437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 10/22] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:24.372862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11c83-9245-44ee-b38e-5005e6839b07 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.373278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11c83-9245-44ee-b38e-5005e6839b07 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.373445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11c83-9245-44ee-b38e-5005e6839b07 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.374975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 13/23] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:24.381519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a5af-8615-45d1-acc5-d2b6a38af359 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.381815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a5af-8615-45d1-acc5-d2b6a38af359 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.382100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a5af-8615-45d1-acc5-d2b6a38af359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.385714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.386006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.386289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.418538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:24.419751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:24.420355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-33b0a5af-8615-45d1-acc5-d2b6a38af359 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:24.421555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-33b0a5af-8615-45d1-acc5-d2b6a38af359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:24.504650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 11/24] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 20 06:53:24.506504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15d7cb-7ac8-4a71-b1ff-8f57ef6cf2c1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.507024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15d7cb-7ac8-4a71-b1ff-8f57ef6cf2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.507501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15d7cb-7ac8-4a71-b1ff-8f57ef6cf2c1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.513079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 12/25] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:24.514795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.515292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.515898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.533102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:24.535180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:24.562384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 14/26] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:24.563436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:24.593972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:24.646308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 13/27] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:24.646841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:24.692523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:24.762080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68d172-ea93-4423-8d13-95a6ee57a5f0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:24.786960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e09da-b2e0-41a8-ba4e-5518f3714858 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:24.832445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:24.833811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.834243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.834820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.842453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:24.843586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:24.844069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:24.844589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:24.862608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:24.866304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:24.868009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:24.874622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:24.875470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:24.876267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:24.887724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 14/28] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:24.889338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:53:24.910919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:24.911765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:24.912626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:53:24.912626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:53:24.912626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:53:24.912626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:24.914927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 15/29] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:24.916051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.021259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.032605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea969c72-864c-4b33-8d73-0cbc1d754b3c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.080007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.091843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.093058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.093350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.093635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.094882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:25.095208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.097995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.109059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 16/30] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:25.109883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.125537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43afd93-6a8a-47ba-8920-891ce1f8594c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.171087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.171686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.171976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.172284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.173045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.174506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.233004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94d6f-1972-4603-a371-a3781124958a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.268960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.269992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.270492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.270965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.271887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:25.272416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.274022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724c514-c3ff-4712-b324-7ba9ba43685b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.293146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 17/31] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:25.294772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08effbce-b5ee-44ce-bfba-b3b40ee45d57 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.295274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08effbce-b5ee-44ce-bfba-b3b40ee45d57 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.295764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08effbce-b5ee-44ce-bfba-b3b40ee45d57 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.297220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 18/32] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:25.302487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08effbce-b5ee-44ce-bfba-b3b40ee45d57 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.312442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 15/33] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Jun 20 06:53:24 2022] POST /identity/v3/roles => generated 219 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:25.313431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.359837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.420023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08effbce-b5ee-44ce-bfba-b3b40ee45d57 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.427378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.434163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eac2a6-2080-464f-a818-11528cc8fc81 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.474216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.475770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.476365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.477041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.478144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.478829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.479013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.479116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.479398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.480425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.481259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81792bc-a06e-4aac-8193-96aa96507f53 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.482137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.483348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 19/34] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:25 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:25.485266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.485790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.486364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.506381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:25.507937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:25.534252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 16/35] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:53:25 2022] POST /identity/v3/projects => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:25.539198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.608734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 20/36] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:25.661660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.681109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8aa2e-7f7b-4084-a97f-5757b80f9268 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.752805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.753620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.753908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.754187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.756998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.758478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.760194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 17/37] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:25.765698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:25.775601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.799125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:25.860014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf1d41-e032-47ea-a4d9-16d8fe763298 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.902238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.903683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.904216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.904389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581490b-2e38-4063-bc49-88eb968bd498 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:25.904876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.905958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.907696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363ef3e-e6bb-41f6-b4ff-40937f76d5ae None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.941222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:25.942014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.942303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.942592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] PATH_INFO: `/v3/domains/default/users/1ecb0c12822b4be89d3c7e17c8adfe7e/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.944930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1ecb0c12822b4be89d3c7e17c8adfe7e, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:25.946391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63f87c-bbc0-4167-be4c-7e25fedf5ebb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:25.962676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 18/38] 10.222.0.23 () {58 vars in 1361 bytes} [Mon Jun 20 06:53:25 2022] PUT /identity/v3/domains/default/users/1ecb0c12822b4be89d3c7e17c8adfe7e/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:53:25.964126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b664-54a9-44e1-a302-4bbdff7822c3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.964440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b664-54a9-44e1-a302-4bbdff7822c3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.964724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9b664-54a9-44e1-a302-4bbdff7822c3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.966146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 19/39] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:25.967243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77602e8-59b1-46a8-9de8-1c2c395d65cb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.967541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77602e8-59b1-46a8-9de8-1c2c395d65cb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.967821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77602e8-59b1-46a8-9de8-1c2c395d65cb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.969321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 20/40] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:25.972166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:25.972509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:25.972780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:25.990671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:25.992279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:26.045228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 21/41] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Jun 20 06:53:25 2022] POST /identity/v3/roles => generated 225 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:53:26.045228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:26.045228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:26.045228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:26.067314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:26.069953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:26.075578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 21/42] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:26.101214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:26.113516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:26.169429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:26.194601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:26.236785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:26.238363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa68147-e640-42e8-9490-81a4e0aea2a0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:26.287523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:26.288660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:26.289176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:26.289454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:26.290274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:26.291710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:26.308639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Jun 20 06:53:26 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:26.428539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd53ab-3bd8-4524-9321-2d1d4acef9b8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:26.466621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:26.467352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:26.467641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:26.467924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:26.469147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:26.470739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74762dfc-9d8c-4322-9a00-4d1b965b4ad2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:26.472447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:26.478166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:26.546088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:26.582089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6a22a-7b6a-4a8d-bd4d-5534798367c6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:26.614306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:26.614959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:26.615244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:26.615520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:26.616267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:26.617724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9202b-825d-4784-a714-035e7f2e1395 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:26.643813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:53:26 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:53:28.105293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cba0-bf90-4925-96b2-502af290987d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.105801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cba0-bf90-4925-96b2-502af290987d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.105885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8cba0-bf90-4925-96b2-502af290987d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.107227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:28.112542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.112976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.113119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.129449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:28.130639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:28.196092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:28.218623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8cba0-bf90-4925-96b2-502af290987d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.318173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8cba0-bf90-4925-96b2-502af290987d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:28.320470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:28.351928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:28.352810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.353095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.353372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.364087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:28.371779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:28.372870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:28.373772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:28.379331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.427076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:28.476499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c90609-3fbd-4897-a095-1b81569b82a7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:28.508645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:28.509302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.509587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.509867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.510613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:28.510920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:28.512329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:28.523617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 25/50] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:28.529057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.627733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efdc98e-6f95-45e6-9165-e61167cf5418 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:28.660488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:28.661353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.661634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.661908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.663085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:28.664568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaed5fc-2c04-4fb0-9b3a-fa0e72ed920f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:28.666408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 26/51] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:28.671561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.675127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe04b2d-2829-413a-b9c6-09dc9e69ff3b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.675420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe04b2d-2829-413a-b9c6-09dc9e69ff3b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.675700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe04b2d-2829-413a-b9c6-09dc9e69ff3b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.677033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 27/52] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:28.681975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.682268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.682554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.701128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:28.702306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:28.728086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:28.773738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 28/53] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:28.798092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.888073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b14825-c5dd-4313-98b4-889847171a90 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:28.898622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:28.902182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5b05c-b7fd-450d-bcc4-5f7624af97a2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:28.923343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:28.924696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.924982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.925269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.926446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:28.927894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:28.929557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 26/54] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:28.935349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.935647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:28.936461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:28.936745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:28.937030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:28.938191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:28.940283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:28.941888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 29/55] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:28.947290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:28.977367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:29.032731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e927-3a7d-4e49-9786-0a19802b0720 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:29.054307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d43614-d5e4-4c8a-9e59-ddf05282406b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:29.066584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:29.067239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:29.067525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:29.067807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:29.068673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.domain.id=default, user.id=1ecb0c12822b4be89d3c7e17c8adfe7e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:29.068988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:29.070433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa7231-dbbe-4350-bf6c-b41fa7b88b2a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:29.078549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 27/56] 10.222.0.23 () {56 vars in 1322 bytes} [Mon Jun 20 06:53:28 2022] GET /identity/v3/role_assignments?user.id=1ecb0c12822b4be89d3c7e17c8adfe7e&scope.domain.id=default&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:29.087895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:29.088618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:29.088914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:29.089202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:29.089945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:29.091764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2067b9-0920-40ad-b710-691c7fa3dcdd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:29.121721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 30/57] 10.222.0.23 () {60 vars in 1140 bytes} [Mon Jun 20 06:53:28 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:53:31.182864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4506cc-cd32-4f41-a6ba-2e80f05a036c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:31.183375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4506cc-cd32-4f41-a6ba-2e80f05a036c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:31.183457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4506cc-cd32-4f41-a6ba-2e80f05a036c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:31.184838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 28/58] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:31.190051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:31.190449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:31.190641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:31.207727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:31.208954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:31.274509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 31/59] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:31.297977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4506cc-cd32-4f41-a6ba-2e80f05a036c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:31.341976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:31.405955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4506cc-cd32-4f41-a6ba-2e80f05a036c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:31.444519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:31.445241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:31.445529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:31.445811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:31.446986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:31.448506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3175eecc-a4a3-44ed-9508-738d37b24f5d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:31.450137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 29/60] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:31.456306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:31.470106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:31.555682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc001db4-500a-4c5e-bd3f-cf019d1f7768 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:31.589152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:31.589812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:31.590099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:31.590379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:31.591102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:31.592580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79163bc5-59f9-4bc8-ab30-5b642d0d5d39 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:31.617552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 32/61] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:31 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:34.657831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790d85-7a3c-481c-8097-529c47401afb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:34.660501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790d85-7a3c-481c-8097-529c47401afb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:34.660501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec790d85-7a3c-481c-8097-529c47401afb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:34.660501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 30/62] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:34.665719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:34.666507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:34.667034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:34.688184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:34.690373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:34.744969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1f7f-ccba-4f05-b82c-b45915c138f3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:34.745576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1f7f-ccba-4f05-b82c-b45915c138f3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:34.746072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c1f7f-ccba-4f05-b82c-b45915c138f3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:34.747627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 31/63] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:34.755570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:34.756194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:34.756688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:34.783665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:34.787344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:34.788747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:34.802297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 33/64] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:34.825851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:34.922499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 32/65] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:34.950733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:34.959566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c5668c-615c-40b9-8631-e2aa49cf23a3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:34.999069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.000234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.000526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.000808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.013111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:35.022427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:35.023283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:35.024155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 34/66] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:35.031024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.044470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.122476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.138811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d0c19-83d3-4e3c-b2e6-1d2bd5d11e96 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.171589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823bb2d2-fe29-4bd3-b720-b3c58d73bea0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.183802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.184850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.185140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.185423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.198948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0819f3d9-f3b2-4c59-a98e-463e47aa352a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:35.207214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:35.208087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:35.208936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 33/67] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:35.209577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.209668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.210202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c25f4-5685-491b-9ae9-08d069a1b75f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.210300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.210377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c25f4-5685-491b-9ae9-08d069a1b75f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.210444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.210509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c25f4-5685-491b-9ae9-08d069a1b75f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.210962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:35.211273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.211745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 34/68] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:35.213071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3a273-b81d-4d26-8267-d03d6bc53e68 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.216984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06ebd5-039a-44f1-a359-3564995afc89 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.217273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06ebd5-039a-44f1-a359-3564995afc89 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.217555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06ebd5-039a-44f1-a359-3564995afc89 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.218982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 35/69] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:35.219580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06ebd5-039a-44f1-a359-3564995afc89 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.221864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 35/70] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:35.223314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ece44-e24c-48e8-8b55-2ac378755260 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.224200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ece44-e24c-48e8-8b55-2ac378755260 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.224738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ece44-e24c-48e8-8b55-2ac378755260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.241449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-962ece44-e24c-48e8-8b55-2ac378755260 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:35.242653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-962ece44-e24c-48e8-8b55-2ac378755260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:35.315307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 36/71] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:35.316994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.317288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.317573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.338208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.339521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:35.340831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:35.343646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06ebd5-039a-44f1-a359-3564995afc89 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.379707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.380417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.380711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.380996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.381793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:35.382112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.383751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.392786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 36/72] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:35.393841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.398693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.420270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 37/73] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:35.429703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.511632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e087f9-d6fb-48e4-a614-115c52a552a8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.546521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.547399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.547687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.547969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.549434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5de7-7b27-44bd-91dc-5e84c0225cc3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.562816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.564320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.570350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:35.570350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.589617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.590362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.591421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.591709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.592045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.603172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:35.611791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:35.612705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:35.613465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:35.614137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.657897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.670124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1021c12-80e0-4b2c-a1d4-e6187147514b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.709773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.710656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.710946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.711228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.712477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.713950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.715581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 38/76] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:35.716374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.719000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13437021-d9ce-4801-b4c3-37dc32f48595 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.764949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.765668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.765962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.766248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.776677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0d45d677-c0bf-4f6c-bf52-e4e0e7fc4cdd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:35.784716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:35.785806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:35.786604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 39/77] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:35.787247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ff863-094c-42c6-ab4b-b2f55f9f5676 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.787491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ff863-094c-42c6-ab4b-b2f55f9f5676 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.787743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ff863-094c-42c6-ab4b-b2f55f9f5676 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.788988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 40/78] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:35.792904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ff863-094c-42c6-ab4b-b2f55f9f5676 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:35.815740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.828006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ba03-d2f1-497d-8925-cc0491527b80 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.861437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.862084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.862395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.862671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.890555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:35.895234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ff863-094c-42c6-ab4b-b2f55f9f5676 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:35.928346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:35.928659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.929005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:35.929294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:35.929575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:35.930179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.930319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:35.930632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:35.932013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:35.939853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 41/79] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:35.944377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.016857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.033510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 39/80] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:36.034337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.043908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d5096-5a21-41a2-b7d3-57cd6b17ea94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.076998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.077642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.077933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.078211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.107941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.130569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bfcdbd-7eee-415f-8d72-2d2b18d5fc58 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.145271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.146757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.164528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.165183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.165466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.165740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.166477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:36.166789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.168171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1fc65-ee2e-438a-a71a-8a19201cff3d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.176474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.177683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.177978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.178257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.200163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:36.201387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:36.245779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.247572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 42/82] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:36.252897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.271782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 41/83] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:36.272386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.310260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.348033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284b5994-b99d-4e33-8960-0bd0f7b5e86e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.382606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.383328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02431c-c943-43d8-a0e2-da8f52d52371 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.383457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.383750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.384031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.385276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.386732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.388398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.389159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.419479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.420370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.420655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.420930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.422107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.424142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.425821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.426570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.475870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.487064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678440a9-4ec1-4767-b4e9-2e891ed41852 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.515188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.521630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.522560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.522789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.523090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.524344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.525801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.527231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298296fd-d8ca-4979-a6eb-b3ee2e424734 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.531899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.531899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.561551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.562403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.562691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.562970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.564321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.565766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.567497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.568287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.630817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014a1b0-a34f-4fd8-aaea-678a84e05127 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.664115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c65cd-54ca-48ad-a0f7-a6f805f986e7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.664280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.665102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.665391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.665671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.666855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.668332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.669936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 45/88] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.670690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.685641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.706348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.706999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.707289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.707576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.739721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.774296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1fcee-49b1-40f4-a2cc-5e86e6891f68 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.774465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.775866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.808966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.809661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.809950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.810241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:36.811081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:36.811401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:36.812886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:36.818279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 46/89] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:36.819064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.888954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 44/90] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:36.889837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:36.913683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.938410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecb75b-6c83-4538-a9fa-ae220b47c688 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:36.972254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:36.979080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:36.979856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:36.980163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:36.980447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.005272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5409d-ed72-4021-a061-960458efcb43 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.045006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.045894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.046457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.046748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.047591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=1ecb0c12822b4be89d3c7e17c8adfe7e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:37.047912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.050117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.051896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.053420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.057092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/role_assignments?user.id=1ecb0c12822b4be89d3c7e17c8adfe7e&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:37.058054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:37.172474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 47/92] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:37.173024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:37.185004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:37.207335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:37.208451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af34d3-d1d0-45e9-b370-bf4fdb0e2c0e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.250369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.251720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.252146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.252554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.253613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.255236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.287269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 46/93] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:37 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:37.288610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:37.301375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65bed2-91a7-48fb-b296-d66fba3306be None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.342937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.344289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.344785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.345898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.347363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.349197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.351325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 48/94] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:37 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:37.359112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:37.421913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62235e16-62a3-4252-bdbe-005cb80150ec None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.467989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.470522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.470522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.470522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] PATH_INFO: `/v3/projects/68d506976e124156bbf8e475ec42db1d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.471515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] RBAC: Authorizing `identity:get_project(project_id=68d506976e124156bbf8e475ec42db1d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.473293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.475329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 47/95] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:37 2022] GET /identity/v3/projects/68d506976e124156bbf8e475ec42db1d => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:37.481927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:37.504515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:37.583222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:37.589058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bebe4c-1211-48fa-a574-d3a8c7b5c62a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.618549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09fc3-0e91-46b9-b2ab-4822e4617d4c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:37.628652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.629572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.630091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.630572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.631628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:37.632205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.633989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8b41c-b9aa-428a-99a2-63055ec63c01 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.640252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:37 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:37.665434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:37.666586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:37.667295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:37.667995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:37.669168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=68d506976e124156bbf8e475ec42db1d, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:37.669727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:37.671444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250aea87-4ef1-4f0c-b998-76c26a22d843 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:37.679021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:37 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=68d506976e124156bbf8e475ec42db1d&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:39.395452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424299-7212-40c8-9039-6c126b9edfd0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.395986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424299-7212-40c8-9039-6c126b9edfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.396621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7424299-7212-40c8-9039-6c126b9edfd0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.399357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:39.405830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.406306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.406692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.419016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d113365-998d-404e-b318-387ffb898200 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.419570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d113365-998d-404e-b318-387ffb898200 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.420063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d113365-998d-404e-b318-387ffb898200 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.421634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 51/99] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:39.427313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.427822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.428333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.437705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:39.446514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:39.448887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:39.450345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:39.574740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:39.581961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 52/100] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:39.587448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 49/101] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:39.627157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:39.628659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:39.836200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:39.846526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d7d80-07b2-4d16-b108-8ab9a01143b7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:39.859817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c1339-debc-4a8d-82c6-c31a753ad339 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:39.926445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:39.927263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.927557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.927839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.948820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:39.949637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:39.949932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:39.950219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:39.951388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:39.959792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:39.960661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:39.961455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:39.969289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 50/102] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:39.969406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:39.977298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:39.985807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:39.986665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:39.987820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:39.993131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.122454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:40.166987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ca5902-29f2-4f0f-8dce-18279bd53420 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.221343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.221956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.222248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.222530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.223314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:40.223633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.225153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.246345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 54/104] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.250422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e8d9e-6a78-49a2-996f-b434a1c4952f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.252372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.293719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:40.325607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.326295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.326584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.326868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.327631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:40.327944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.329432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.346597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 51/105] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.357374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.502229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4edd8-827a-4531-b189-954113483e55 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.534712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87f475-d947-40e5-a1e9-acb7ed635e49 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.558381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:40.574682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.575565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.575856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.576151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.577581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.577686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.578577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.578866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.579149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.579761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.581765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 52/106] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:40 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.584961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.586412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.587013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.588321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 55/107] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:40 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.595572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.721659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:40.733727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366906cb-3d54-47b0-884b-0fa208d4aa89 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.747262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37771c80-f269-48be-9461-b72e40617f06 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.770091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.771294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.771767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.772319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.773746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.775421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c27a3-28cd-4c99-93a3-d880328141a1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.777394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 53/108] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:40 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.778763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e2c3-d2f9-4070-aec7-ec1c6fa1a081 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.779276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e2c3-d2f9-4070-aec7-ec1c6fa1a081 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.779758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0e2c3-d2f9-4070-aec7-ec1c6fa1a081 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.781239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 54/109] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:40.782819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e905-5368-4d7d-87be-99d9c1f21c87 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.783606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e905-5368-4d7d-87be-99d9c1f21c87 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.784302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e905-5368-4d7d-87be-99d9c1f21c87 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.785830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 55/110] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:40.790636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6e905-5368-4d7d-87be-99d9c1f21c87 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:40.794224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.795089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.795378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.795654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.798421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.799896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0066a-b7ed-4298-91f8-a9fc7ec6d73a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.801797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 56/111] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:40 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:40.805311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45da5-54b7-4e21-8385-796d8cb4d463 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.805836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45da5-54b7-4e21-8385-796d8cb4d463 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.806322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb45da5-54b7-4e21-8385-796d8cb4d463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.826681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-abb45da5-54b7-4e21-8385-796d8cb4d463 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:40.827865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-abb45da5-54b7-4e21-8385-796d8cb4d463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:40.862618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:40.900758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 57/112] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:40.909523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.909736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.910256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.917284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6e905-5368-4d7d-87be-99d9c1f21c87 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:40.941533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:40.942718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:40.958251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:40.959523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:40.960061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:40.960609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/users/1ecb0c12822b4be89d3c7e17c8adfe7e/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:40.962877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, user_id=1ecb0c12822b4be89d3c7e17c8adfe7e, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:40.964788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:40.981145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 56/113] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:40 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/users/1ecb0c12822b4be89d3c7e17c8adfe7e/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:40.983227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.025816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 58/114] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:41.026931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.049072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.147398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813ccbc7-051f-4fa2-9020-1f0a0fb90c8e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.166046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.194517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.196076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.196855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.197345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/users/abeac7d1d42343afbdb5c2951af21e03/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.199945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, user_id=abeac7d1d42343afbdb5c2951af21e03, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:41.202491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:41.218169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6e15-2615-4773-83fa-76aa1ba0b6ae None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.227692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 57/115] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:40 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/users/abeac7d1d42343afbdb5c2951af21e03/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:41.228618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.296880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.297636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.297974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.298758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.321152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b0372b19-3c81-4300-aae0-1bcfff79c230 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:41.337592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:41.338534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:41.339313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 59/116] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:41.340149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d44d-1b90-4664-b9d1-7d2f9581b865 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.344154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d44d-1b90-4664-b9d1-7d2f9581b865 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.344154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d44d-1b90-4664-b9d1-7d2f9581b865 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.344154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 60/117] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:41.345897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117d44d-1b90-4664-b9d1-7d2f9581b865 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.432553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.460730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a77b3-ebe1-4891-b1bd-d81338b2d835 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.471958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117d44d-1b90-4664-b9d1-7d2f9581b865 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.497379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.498185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.498474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.498756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.510460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:41.513676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.514363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.514652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.514933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.515701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:41.516013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:41.518802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-76af64a8-0acd-47eb-869f-232140e67a91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:41.519670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:41.520600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:41.521367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 58/118] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:41.522518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.524138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.524138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.530683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 61/119] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:41.531579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.541773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:41.543072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:41.573725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.618627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 59/120] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:41.618918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.670139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bd48c7-b520-4303-87d4-e350bcf41fe4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.710521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.717676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.718564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.719014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.719458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.721588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:41.722225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:41.724198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fde69f-7d78-44d3-a1d1-a3c4ac9ae7eb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:41.735538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361ef33-0f3d-48eb-9ea5-17035156e740 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.737743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 62/121] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:41.740241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286ed49-fe3f-45e7-b7c9-2052fe1bdd95 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.740739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286ed49-fe3f-45e7-b7c9-2052fe1bdd95 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.741207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286ed49-fe3f-45e7-b7c9-2052fe1bdd95 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.742747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 63/122] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:41.743631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286ed49-fe3f-45e7-b7c9-2052fe1bdd95 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.782284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.783305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.783668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.784055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.786037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:41.787884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:41.795170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 60/123] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:41.795170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.838367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:41.888091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286ed49-fe3f-45e7-b7c9-2052fe1bdd95 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.915501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c012d5-7798-40d8-950a-e2ac5b75d33f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:41.933318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.933318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.933318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.933318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.944802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:41.953362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a80fd721-657d-438d-9b9e-be42b8d0c3c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:41.953863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:41.955387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:41.956264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:41.956888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.956888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.956888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.957081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 64/124] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:41.957081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:41.957406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:41.959500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:41.959619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:41.960379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:41.962636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 61/125] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:41.963527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:41.977458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:41.978910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:42.044085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.069939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 65/126] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:42.071177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.077505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa776d-5efe-4cf7-a187-e0c339e44895 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.079735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.130531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.131418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.131774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.132205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.133626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.135210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.137026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 62/127] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:41 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:42.137863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.191550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a66516-5de0-48bb-bdbb-2a9e3a411b18 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.227919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.228938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.229443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.229899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.231285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:42.231788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.233930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.244363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5eb6d0-0f03-4a80-950c-cf01d493e6dc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.245761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 66/128] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:42.247863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.281429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.282481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.282833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.283236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] PATH_INFO: `/v3/projects/68d506976e124156bbf8e475ec42db1d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.284504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] RBAC: Authorizing `identity:get_project(project_id=68d506976e124156bbf8e475ec42db1d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.286171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.288361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 63/129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:42 2022] GET /identity/v3/projects/68d506976e124156bbf8e475ec42db1d => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:42.289081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.310334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.332299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.395261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d66f05-d8b1-4928-a565-3ac83b8cc67a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a79d43-7437-46c6-8a7e-131f134870f4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.456855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.458418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.459015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.459766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.461310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.462245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.462607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.463072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.463521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/users/abeac7d1d42343afbdb5c2951af21e03/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.465405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.466249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, user_id=abeac7d1d42343afbdb5c2951af21e03, role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.467844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.469031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 67/130] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:42.477165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.494000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 64/131] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:42 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/users/abeac7d1d42343afbdb5c2951af21e03/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:42.494606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.664512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.669715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3206c-3e30-4bba-8102-c453f50faacc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.712928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b1643-db03-4516-ae85-1d850b6c702d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.714097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.716346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.716852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.717328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] PATH_INFO: `/v3/projects/68d506976e124156bbf8e475ec42db1d/users/abeac7d1d42343afbdb5c2951af21e03/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.720596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d506976e124156bbf8e475ec42db1d, user_id=abeac7d1d42343afbdb5c2951af21e03, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.722742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.728936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:42.747803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 65/132] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:42 2022] PUT /identity/v3/projects/68d506976e124156bbf8e475ec42db1d/users/abeac7d1d42343afbdb5c2951af21e03/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:42.748992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.756929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.757891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.758183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.758465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] PATH_INFO: `/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.774635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] RBAC: Authorizing `identity:get_user(user_id=803ea7d1e34d4dab81772e6ebbbb8d2f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.776467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.778064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 68/133] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:42 2022] GET /identity/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f => generated 289 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:42.784957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:42.901297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c496ba9-6d43-4874-97d2-4991cac57259 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.909001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b60b65-8e46-47a5-9cfb-240c4d387f47 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:42.951293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.951293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.951293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.951293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.970615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:42.971655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:42.972185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:42.972819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:42.973898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:42.980129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:42.998043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:43.028544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:43.030095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:43.037634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 66/134] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:53:42 2022] POST /identity/v3/projects => generated 302 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:43.138740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 69/135] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:43.144813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:43.164007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:43.284619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f20cc2-51fb-4392-b468-5a4ba5368a25 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:43.324914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:43.326210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.326745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.327261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.328821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:43.330633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5262f551-bcc8-49ca-a18e-3e56ab456327 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:43.332629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 67/136] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:43 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:43.338355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:43.441207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:43.484563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa811a3-a184-49b1-a140-3127c1744cc8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:43.527371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:43.528039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.528358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.528644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.529498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=803ea7d1e34d4dab81772e6ebbbb8d2f) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:43.529818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:43.531284 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b499910-0d92-4af6-8a66-9516c47edab6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:43.542084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 70/137] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:43 2022] GET /identity/v3/role_assignments?user.id=803ea7d1e34d4dab81772e6ebbbb8d2f&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:43.560777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9d3-cdb7-402d-aea1-abf06f7b2cd4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.561113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9d3-cdb7-402d-aea1-abf06f7b2cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.561345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eec9d3-cdb7-402d-aea1-abf06f7b2cd4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.563114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 68/138] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:43.570301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa6da-a498-41d7-b133-191d3146a060 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.570617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa6da-a498-41d7-b133-191d3146a060 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.570888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013aa6da-a498-41d7-b133-191d3146a060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.597206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-013aa6da-a498-41d7-b133-191d3146a060 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:43.598423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-013aa6da-a498-41d7-b133-191d3146a060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:43.716312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 71/139] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:43.767041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdc54-1bc1-4575-910e-819e53c0ca0e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.767578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdc54-1bc1-4575-910e-819e53c0ca0e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.767578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcdc54-1bc1-4575-910e-819e53c0ca0e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.769305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 69/140] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:43.775036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.775336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.775625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.775792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdc54-1bc1-4575-910e-819e53c0ca0e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:43.805246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:43.806400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:43.865638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:43.891352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcdc54-1bc1-4575-910e-819e53c0ca0e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:43.927663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:43.928496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:43.928783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:43.929064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:43.940975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:43.949670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:43.951174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:43.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:43.952846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 70/141] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:43.960061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:43.969485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 72/142] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:44.031975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.157738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113ee01-9c58-4aa7-b5ca-dbb8fcc3ba26 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.170252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7e3b72-ae1b-4945-90a0-d82cb26e0cde None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.199266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.200354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.200839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.201776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.214213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:44.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:44.224333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:44.225264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:44.227506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 73/143] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:44.234223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.272834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.273520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.273820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.274110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.274877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:44.275193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:44.276639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:44.288748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:44.295388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 71/144] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:44.307456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:44.317147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.362622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7383f19-3cf6-4e26-a1b4-1fa8caaefbbf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.402115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.403126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.403566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.404022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.405002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:44.405482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:44.407569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:44.417571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 74/145] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:44.424439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.548919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cfc8e-e36b-447f-89ee-cf7d944bcae9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.569898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:44.590843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.591744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.592032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.592933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.594288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:44.596719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:44.598681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 75/146] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:44.605058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.634060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4983a4f1-37b7-4b0b-9b50-c91db00122f0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.722633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451322aa-0579-4e82-bbff-ac54655d915c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:44.740798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.741695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.741988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.742278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] PATH_INFO: `/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.743511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] RBAC: Authorizing `identity:get_user(user_id=1ecb0c12822b4be89d3c7e17c8adfe7e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:44.744995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:44.746806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 72/147] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/users/1ecb0c12822b4be89d3c7e17c8adfe7e => generated 247 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:44.761969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:44.763709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:44.764411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:44.764696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:44.764977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:44.827230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:44.844015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:44.848099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:44.945813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:44.988291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 76/148] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:53:44.995844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:45.034515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21da46f-1ebd-453a-9f58-633e3164c61d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:45.117024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:45.117679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.117968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.118250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.176167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:45.203110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7974d-9e02-4a1b-b8dc-c2233765284e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:45.227401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:45.229232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:45.281744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:45.283190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.283482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.283764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.285031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:45.286526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:45.288271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 77/149] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:45.309129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:45.332577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:45.384358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 73/150] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:45.389928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:45.500454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916b816-6da1-4ec4-b4e6-ca75bda268c6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:45.526245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25587ab-a493-4843-8e88-214df34ff7e7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:45.541457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:45.542233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.542514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.542841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.545035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:45.547652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:45.553986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 74/151] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:45 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:45.555768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:45.593441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:45.604187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:45.604855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.605147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.605428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.606279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:45.606597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:45.608058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7037ae-a429-4aa8-89b1-189bd22a56c0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:45.622387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 78/152] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:45 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:45.685942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8e85a-2fb5-417d-be85-6529b8cf606a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:45.739845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:45.740545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.740834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.741115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.741973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=1ecb0c12822b4be89d3c7e17c8adfe7e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:45.742290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:45.743787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4a809-e81f-4e97-8d7e-ffc17c310d7b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:45.757395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 75/153] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:45 2022] GET /identity/v3/role_assignments?user.id=1ecb0c12822b4be89d3c7e17c8adfe7e&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:45.816371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a683a7-e2bc-4843-9335-ece899fd2995 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.816853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a683a7-e2bc-4843-9335-ece899fd2995 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.816853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a683a7-e2bc-4843-9335-ece899fd2995 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.818278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 79/154] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:45.824248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466e35-6094-45a6-ad4d-1a1b629ef1bc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:45.824568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466e35-6094-45a6-ad4d-1a1b629ef1bc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:45.824828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b466e35-6094-45a6-ad4d-1a1b629ef1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:45.845574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0b466e35-6094-45a6-ad4d-1a1b629ef1bc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:45.846814 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0b466e35-6094-45a6-ad4d-1a1b629ef1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:45.935502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 76/155] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:45.972283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a683a7-e2bc-4843-9335-ece899fd2995 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:45.997542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.027460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfb53-64b7-4583-95ce-87f2ea741d76 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.027957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfb53-64b7-4583-95ce-87f2ea741d76 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.028028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbfb53-64b7-4583-95ce-87f2ea741d76 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.029631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 77/156] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:46.035521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.035800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.036078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.064356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:46.065556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:46.099497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.126515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a683a7-e2bc-4843-9335-ece899fd2995 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.152878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 78/157] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:46.163843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.164691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.164980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.165257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.175978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:46.180210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:46.184247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:46.186606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:46.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 80/158] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:46.197078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.313908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.317067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4439-4ecb-4b5e-861d-4b97e077cf7c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.341799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41676f4-419a-4f90-9c53-e6ccdcad7258 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.363731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.364959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.365423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.365871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.379668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:46.387942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:46.388922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:46.389706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:46.392097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 79/159] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:46.394630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.395861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.396452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.397034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.398517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:46.399150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.401115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.401873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.427237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 81/160] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:46.429199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.431219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.517472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc90bfe-5c64-4791-86c4-084a7ebad26e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.550836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638336d-ba4f-41d2-927f-00312546c16a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.552490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.553160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.553447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.553730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.554484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:46.554791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.556195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a4e02-34af-4ffb-85de-346cec6c090e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.564635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 80/161] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:46.565726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ea92c-c141-47c2-8112-919140b61a99 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.566016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ea92c-c141-47c2-8112-919140b61a99 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.566297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ea92c-c141-47c2-8112-919140b61a99 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.567651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 81/162] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:46.573204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.573490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.573766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.590546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:46.591772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:46.598185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.599069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.599355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.599634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.600879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.602376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.608282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 82/163] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:46.608282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.618893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.666193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 82/164] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:46.667101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.680876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.713571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2b7f8-42bd-4af7-b2fc-33e5b4d89bca None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.751631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.752570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.752849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.753128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] PATH_INFO: `/v3/users/abeac7d1d42343afbdb5c2951af21e03` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.754353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] RBAC: Authorizing `identity:get_user(user_id=abeac7d1d42343afbdb5c2951af21e03)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.756027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c1355-b590-427b-9577-4e9c82b47e5b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.757987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 83/165] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/users/abeac7d1d42343afbdb5c2951af21e03 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:46.759247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfc1a-5078-4a17-9b4e-bd1a1f3283cd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.759522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfc1a-5078-4a17-9b4e-bd1a1f3283cd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.759797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dfc1a-5078-4a17-9b4e-bd1a1f3283cd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.761046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 84/166] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:46.764575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dfc1a-5078-4a17-9b4e-bd1a1f3283cd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.786493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd6676-6661-4a21-bcef-ea37a85473d6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.819498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.820211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.820496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.820774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.855119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.878571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dfc1a-5078-4a17-9b4e-bd1a1f3283cd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:46.885943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.887402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b94d5f-49a2-4e38-9630-c713db0ec458 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.901117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:46.913598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:46.914382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.914666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.914950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:46.916123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:46.917568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:46.919186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 85/167] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:46.919943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:46.985270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 83/168] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:46.986534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:46.986821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:46.987089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.002838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:47.004007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:47.019922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f0b58-d738-4e6d-94af-72a7cdc4d576 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.052270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.052911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.053200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.053481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.067517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 84/169] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:47.068333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.072988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.112782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.131755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.133200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.168016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef475756-15ea-4195-8f3c-417d23dd9099 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.201420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.202078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.202356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.202626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.203325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.205315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.224497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 85/170] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:47 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:47.225300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.232206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 86/171] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:47.237210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.290811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.349674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.352359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d55e4d-d66c-40f6-9569-ea7d9a4bb2e8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.353393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8814f721-06e7-4e6c-9c8b-85d4a56cac24 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.394963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.395684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.395959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.396265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.398995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.400197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.400482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.400756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.403017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.405010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.407089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 86/172] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:47.407766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.412232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:47.419984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:47.421006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:47.421807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 87/173] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:47.423020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.537134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eee9b-ad54-4ad8-893f-595dc0327ec3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.552348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daaecc-2fbe-4728-8f61-4a1396afb8ed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.554854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.583664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.584557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.584835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.585104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] PATH_INFO: `/v3/projects/68d506976e124156bbf8e475ec42db1d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.586289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] RBAC: Authorizing `identity:get_project(project_id=68d506976e124156bbf8e475ec42db1d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.587743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.589477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 87/174] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/projects/68d506976e124156bbf8e475ec42db1d => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:47.590319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.595004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.595972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.596448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.596929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.597953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:47.598465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.600294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.607161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 88/175] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:47.616549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.643501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.732591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bbf6c-11de-4679-8d39-5a38e9107261 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.771837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6b173-78ad-48ed-b9d8-f8fde140c260 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.778242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.778930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.779212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.779490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.780288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:47.780696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.783444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.795494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 88/176] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:47.801662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:47.820918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.821584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.821868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.822143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.823044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=68d506976e124156bbf8e475ec42db1d, user.id=abeac7d1d42343afbdb5c2951af21e03) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:47.823360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.825011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.832239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 89/177] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/role_assignments?user.id=abeac7d1d42343afbdb5c2951af21e03&scope.project.id=68d506976e124156bbf8e475ec42db1d&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:47.921218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:47.926782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef31056-3ac7-4555-a7d6-eeac05ab5353 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:47.970844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:47.971974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:47.972478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:47.972898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] PATH_INFO: `/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:47.974390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] RBAC: Authorizing `identity:get_user(user_id=803ea7d1e34d4dab81772e6ebbbb8d2f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:47.976029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:47.978024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 89/178] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:47.983894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:48.022495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:48.106141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0444d-980e-45a7-b2a9-0a21f2095bac None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:48.144885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:48.145751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:48.146037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:48.146311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:48.147584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:48.149120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f71f1-54cf-4f4a-9829-d9098bf16c31 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:48.150835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 90/179] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:47 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:48.156587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:48.269762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099145f-f6ea-4462-bdf6-376f1796da14 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:48.311727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:48.312662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:48.312961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:48.313774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/users/803ea7d1e34d4dab81772e6ebbbb8d2f/roles/e7f74e1aab7b4bd79af410095f5921a2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:48.315768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, user_id=803ea7d1e34d4dab81772e6ebbbb8d2f, role_id=e7f74e1aab7b4bd79af410095f5921a2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:48.317235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d53c56-2662-4e86-8c7c-38eb5632b560 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:48.336313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 90/180] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:48 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/users/803ea7d1e34d4dab81772e6ebbbb8d2f/roles/e7f74e1aab7b4bd79af410095f5921a2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:49.601869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72718c-deda-48f2-9b8b-26ee1fcfeaa5 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.602383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72718c-deda-48f2-9b8b-26ee1fcfeaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.602458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72718c-deda-48f2-9b8b-26ee1fcfeaa5 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.604055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 91/181] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:49.609496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c0a-38c1-4c89-93bf-124552db82f9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.610452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c0a-38c1-4c89-93bf-124552db82f9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.610621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d8c0a-38c1-4c89-93bf-124552db82f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.628365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8a2d8c0a-38c1-4c89-93bf-124552db82f9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:49.629571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8a2d8c0a-38c1-4c89-93bf-124552db82f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:49.647740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:49.705109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 91/182] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:49.733172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72718c-deda-48f2-9b8b-26ee1fcfeaa5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:49.838481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:49.841128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da1f6c-d181-424e-94eb-4447510dad69 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.841379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da1f6c-d181-424e-94eb-4447510dad69 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.841844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da1f6c-d181-424e-94eb-4447510dad69 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.843323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 92/183] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:49.843826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72718c-deda-48f2-9b8b-26ee1fcfeaa5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:49.848695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e3d2d-5d75-4b27-9149-16d8601b7280 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.848984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e3d2d-5d75-4b27-9149-16d8601b7280 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.849266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e3d2d-5d75-4b27-9149-16d8601b7280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.866583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ab4e3d2d-5d75-4b27-9149-16d8601b7280 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:49.867773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab4e3d2d-5d75-4b27-9149-16d8601b7280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:49.883728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:49.884848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.885454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.885930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.899074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:49.907471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:49.908419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:49.909210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:49.916519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 92/184] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:49.918286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:49.944282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 93/185] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:49.955651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429b1f-0519-496b-bcb4-a94f5f08feb8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.955938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429b1f-0519-496b-bcb4-a94f5f08feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.956246 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20429b1f-0519-496b-bcb4-a94f5f08feb8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.957746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 94/186] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:49.962424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:49.962708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:49.962979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:49.980917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:49.982088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:50.027952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d02a4-ab7d-4d4f-8894-9647d45df6ea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.051777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 95/187] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:50.056987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.065723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.066382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.066672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.066953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.067870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:50.068070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.069437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.077042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.078287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 93/188] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:50.083657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.117841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.164217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e76b8-f59d-47b1-a84b-e2fa716a01d1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.197817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bc3259-ddfa-43ca-9a53-2ddf13eb0717 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.200584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.201393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.201681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.201962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.212890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:50.221789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:50.222673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:50.223465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 96/189] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:50.224139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.238655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.239444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.239726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.240001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.241200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.242645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.244441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 94/190] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:50.245039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.310623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.338585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649b493-87d9-4452-ad5d-6d7aec559abc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.348798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.359736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1309d1-bd0d-460e-9e94-c4e01a45120e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.373701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.374750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.375032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.375344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.389600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.391614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.398142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 97/191] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:50.398142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.400402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.401997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.403027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.403027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.403298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:50.403603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.405000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.418177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 95/192] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:50.418177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.503601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61aeefd-74b3-484a-a4b2-62c155e3fa55 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.519727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde359-f0c9-4abd-9991-1eba2a138fa1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.533681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.539136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.539803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.540088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.540390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.541105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.542544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.552245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.552885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.553170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.553452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.561576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 98/193] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:50 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:50.565066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.575347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.631609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.633919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7bf21-d3a8-4cb1-8901-f9d810c6c211 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.689923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d293-f260-4607-83b0-1d1438902491 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.738470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.739400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.739687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.739955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.741185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:50.742666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb4199-9527-4934-815b-7c4120700bdf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:50.744309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 99/194] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:50.749979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7565f6-9eca-4dc5-bd8f-e40f311e5640 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.750795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7565f6-9eca-4dc5-bd8f-e40f311e5640 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.751110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7565f6-9eca-4dc5-bd8f-e40f311e5640 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.756308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 100/195] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:50.756308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7565f6-9eca-4dc5-bd8f-e40f311e5640 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.758364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 96/196] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:50.759287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.759581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.759859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:50.779334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:50.780559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:50.833480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.854753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:50.884212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 97/197] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:50.884212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:50.890435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7565f6-9eca-4dc5-bd8f-e40f311e5640 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:50.930446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:50.931095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:50.931376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:50.931650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.003578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae3557-ab34-4c22-b36c-d1f2e4b9855e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.023881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.025483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.042616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.043865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.044385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.044923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.046320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.048024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.050318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 98/198] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.051287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.108650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.120075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.159373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 101/199] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:51.164156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.181968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e78b-b167-490f-ac85-5b7ab614173c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.235180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.235985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.236300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.236567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.247401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:51.255616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:51.256690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:51.257686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 99/200] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:51.258452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.276635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bbaf0-3afd-47ee-bcbc-31b6841da3d1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.317075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.318291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.319173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.319602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.320861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=17be58d6d34d4c488c340743b85df57e) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:51.321649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.323619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.332257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 102/201] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/role_assignments?user.id=17be58d6d34d4c488c340743b85df57e&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.333232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.361245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.449786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93bba8f-8fc8-4769-b88b-1a21113d76f7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.469581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154c03c-1106-4692-a3d5-07e0c6bd3c80 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.473283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.516865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.519018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.520843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.521306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.522464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:51.524072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.526803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.539397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.540584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.542240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.542664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.545511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 103/202] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.546321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.550313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.552307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 100/203] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.553387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.558171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.673199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cac06-c1a2-4883-949f-0072e6c90f1a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.714928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.715590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.715868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.716154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.716993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=17be58d6d34d4c488c340743b85df57e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:51.717304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.718747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.722391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.729621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 101/204] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/role_assignments?user.id=17be58d6d34d4c488c340743b85df57e&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.766250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4bbd1-d22b-460f-9668-48ff77fd853f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:51.841618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:51.842526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:51.842810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:51.843086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] PATH_INFO: `/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:51.844919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] RBAC: Authorizing `identity:get_user(user_id=803ea7d1e34d4dab81772e6ebbbb8d2f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:51.846429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:51.848084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 104/205] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/users/803ea7d1e34d4dab81772e6ebbbb8d2f => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:51.853657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:51.918781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:51.989418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6ac9c-55d9-453b-8037-c19f604f1a45 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:52.031234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:52.032494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:52.032778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:52.033055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:52.110441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:52.111931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:52.155488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:52.214359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 102/206] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:52.220634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:52.343736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054579e9-2e91-406b-81e4-127a869c9b51 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:52.377902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:52.378829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:52.379115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:52.379394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:52.380627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:52.382096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5f3a8-4936-4490-91f7-51256e7e0093 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:52.384005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 105/207] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:52 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:52.389723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:52.493558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad347856-7cb0-4ac1-9783-6b7b076dc6e5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:52.529792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:52.530513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:52.530796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:52.531074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:52.531998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, scope.project.id=15f6d807458c48ecb1d58725592095f0, user.id=803ea7d1e34d4dab81772e6ebbbb8d2f) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:52.532343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:52.533789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1ad5ec-ffa6-42ab-9534-746abdd2ebc0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:52.540128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 103/208] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:52 2022] GET /identity/v3/role_assignments?user.id=803ea7d1e34d4dab81772e6ebbbb8d2f&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:52.952642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e129-1cc2-490a-acf5-d0b73b790542 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:52.953130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e129-1cc2-490a-acf5-d0b73b790542 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:52.953259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8e129-1cc2-490a-acf5-d0b73b790542 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:52.954658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 106/209] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:52.962119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:52.962119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:52.962119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:52.976296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:52.982442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:52.983822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:53.029342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc7045-661f-4cc8-917c-ffab4321aba0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.029631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc7045-661f-4cc8-917c-ffab4321aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.029918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc7045-661f-4cc8-917c-ffab4321aba0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.034600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 107/210] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:53.041396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.041687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.041969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.065828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:53.066243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:53.082983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 104/211] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:53.089611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:53.109219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.153268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 108/212] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:53.180092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.216153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee60674-a837-43c5-88eb-183bbda5a824 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.253438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.254811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.255306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.255802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.258795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:53.267962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:53.276670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:53.277617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:53.278724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 105/213] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:53.284812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.292497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5de1b-ac21-4710-b886-75f8f1795ee8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.332749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.333870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.334469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.334936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.336365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:53.338279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:53.340093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 109/214] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:53.346447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.366460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:53.402995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318660a-e0eb-4cda-b279-0f62536ebb02 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.442552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.442552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.442552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.442903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.444191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:53.444191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:53.445665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:53.455237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 106/215] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:53.460518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.476869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53cb8c7-1839-4bdb-b399-762ead7edde1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.514278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.515645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.516660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.517143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.518290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:53.520798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:53.524700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:53.556133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 110/216] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:53 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:53:53.581275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668b1fb-c171-4878-8730-642a130c0289 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.621170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.622082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.622370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.622647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] PATH_INFO: `/v3/users/0bbd0af1acc54978824d3fe702dd202a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.642900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbd0af1acc54978824d3fe702dd202a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:53.644967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aac5fb-56d3-47fe-a89c-5a4f8349aa65 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:53.647065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 107/217] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/users/0bbd0af1acc54978824d3fe702dd202a => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:53.652688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.725669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b8e9-93c7-40b4-9596-a094eef6b07e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.726010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b8e9-93c7-40b4-9596-a094eef6b07e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.726227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b8e9-93c7-40b4-9596-a094eef6b07e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.727943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 108/218] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:53.733150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.733437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.733721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.748138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:53.755563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:53.756804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:53.797236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd428a-5525-493e-bb90-a074a64e181d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:53.846182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:53.847179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:53.847660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:53.848172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:53.849931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 109/219] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:53.882777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:53.934332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:53.936080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:53.950602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.012461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa341575-bf7d-48f3-b2cb-9cd5ee66f0b4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.028501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.052009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.052931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.053283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.053633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.067671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:54.068140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 111/220] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:54.073264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:54.077856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:54.079331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:54.080164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 110/221] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:54.090486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.200597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206eece-ad92-40af-869a-ac8972e20162 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.228203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e056-4eb6-4eae-8fb6-732499e888c1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.237740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.260252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.261136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.261422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.261713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.265363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.266726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.268768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 112/222] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:54.270894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.271806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.272327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.272797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.274080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:54.274475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.274871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.276598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.287473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 111/223] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:54.295518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.352910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.416148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1577c7e-de87-40ec-b724-c885a83de96a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.452786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4a40d-6f2b-4532-b223-6af773809cf0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.460825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.461985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.462274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.462556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.463865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=0bbd0af1acc54978824d3fe702dd202a) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:54.464202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.466258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6aea1-c65c-4234-8898-9cf3d4a95835 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.474716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 113/224] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/role_assignments?user.id=0bbd0af1acc54978824d3fe702dd202a&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:54.476553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874201-570c-4fb8-ae28-98da00ac3bc6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.476849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874201-570c-4fb8-ae28-98da00ac3bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.477128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f874201-570c-4fb8-ae28-98da00ac3bc6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.478777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 114/225] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:54.484726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.485024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.485287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.504042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:54.505359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:54.505495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.506383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.506670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.506947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.513056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.514558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.516966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 112/226] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:54.522877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.580133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 115/227] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:54.602086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.621605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.640469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.727936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d26e1-dfa3-4694-a253-8b282a63895b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.740231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee690b-553d-4137-bd41-90a50256e66e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.764921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.766456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.766743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.767075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.768619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.770198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.772501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 113/228] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:54.777579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.785630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.786526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.786922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.787202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.815897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:54.833077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:54.834041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:54.835392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 116/229] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:54.841889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:54.895540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36ae23-d20d-468d-bc50-1a5a7c47d2c4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:54.898913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:54.939057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:54.939973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:54.940337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:54.940632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/17be58d6d34d4c488c340743b85df57e/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:54.942981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, user_id=17be58d6d34d4c488c340743b85df57e, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:54.944475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:54.973914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 114/230] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:54 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/17be58d6d34d4c488c340743b85df57e/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:54.997602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:55.006482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035251fd-1fea-4fa6-a289-a7594a904301 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:55.045131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:55.045922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:55.046113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:55.046349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:55.047637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:55.047944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:55.049460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:55.058851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 117/231] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:55.065294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:55.226974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3967d-1162-4915-a395-a11df68fc01d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:55.275309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:55.276229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:55.276517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:55.276798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:55.278001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:55.279473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:55.281145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 115/232] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:55 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:55.287219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:55.407242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb2fd2-a968-4648-8b08-786157e88b3d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:55.445217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:55.446089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:55.446371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:55.446649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:55.447867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:55.449357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48696d2e-5fe2-4e9e-8a3c-8971be0bd95f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:55.451370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 118/233] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:55 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:55.456654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:55.516303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:55.621211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a1716-302c-411a-9a49-ab7aa436c6af None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:55.658155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:55.658969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:55.659255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:55.659535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/17be58d6d34d4c488c340743b85df57e/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:55.661886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, user_id=17be58d6d34d4c488c340743b85df57e, role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:55.663398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9cbc-44f2-49e9-8de0-511f440b17ce None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:55.689643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 116/234] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:55 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/17be58d6d34d4c488c340743b85df57e/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:56.296555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265894-5bd5-45a2-b0e3-3c32096109cc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.298099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265894-5bd5-45a2-b0e3-3c32096109cc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.298576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3265894-5bd5-45a2-b0e3-3c32096109cc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 119/235] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:56.305431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1c30-a824-4706-9854-d95609339251 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.305841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1c30-a824-4706-9854-d95609339251 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.306024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f1c30-a824-4706-9854-d95609339251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.329398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f82f1c30-a824-4706-9854-d95609339251 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:56.330587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f82f1c30-a824-4706-9854-d95609339251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:56.401994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa7a5-31eb-431a-b105-68ab018cc7a8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.402644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa7a5-31eb-431a-b105-68ab018cc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.403140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfa7a5-31eb-431a-b105-68ab018cc7a8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.403452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 117/236] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:56.405218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 120/237] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:56.428004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1c30-a824-4706-9854-d95609339251 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:56.431979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.432504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.433001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.454391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:56.466856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:56.468045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:56.526909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:56.574480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f1c30-a824-4706-9854-d95609339251 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:56.630030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 121/238] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:56.631399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:56.632378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.632806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.633250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.634565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:56.636172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:56.638037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 118/239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:53:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:56.643747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:56.656768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:56.795977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04006467-f946-4f34-8f59-e9dd0c0b705b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:56.808493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:56.832727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:56.833584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.833874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.834152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.845653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:56.853931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:56.855044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:56.856186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 119/240] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:56.862499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:56.880845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e67373-0080-4ab2-a8b9-4c32049b934e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:56.948889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:56.949573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.949858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.950137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.950858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:56.952333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0939df6-86e4-4ac8-99da-f67b48b8412f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:56.986645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 122/241] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:53:56 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:53:56.987784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf05af-acae-45fb-bfb0-1553746771f3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:56.988078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf05af-acae-45fb-bfb0-1553746771f3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:56.992713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf05af-acae-45fb-bfb0-1553746771f3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:56.994653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 123/242] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:57.010076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.010739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.011224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.045315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:57.046546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:57.052470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.067608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362750a1-1190-424c-bb79-6fcec714f0fd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.134793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.135402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.135685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.135959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.136758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:57.137066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.138484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.154375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 120/243] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:57.162222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.163453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 124/244] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:57.188100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.190248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.298099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3212-2146-4a0b-833c-9243080f4b8a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.326316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3abdb-4f5b-406c-a140-a92268a1898b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.337189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.338052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.338446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.338728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.348457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.353924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:57.362359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:57.363258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:57.364075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 125/245] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:53:57.364766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.364766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.365630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.365630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] PATH_INFO: `/v3/users/485e5609c19044d3a978abceedcffeb1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.371717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.379453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] RBAC: Authorizing `identity:get_user(user_id=485e5609c19044d3a978abceedcffeb1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.380967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.382850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 121/246] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/users/485e5609c19044d3a978abceedcffeb1 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:57.388360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.472452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.501446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb5312-d724-477b-aad3-c16905782449 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.510496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d69cea-f1fb-414b-a95c-4cbdefe42020 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.548089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.549333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.549819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.550391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.551328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.551539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.551816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.552092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.552639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:57.553141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.554729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.571506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 126/247] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:57.577452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.636259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.637993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f4c2f7-d5cb-410a-b979-db28d953c00d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.671601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.726629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3636f-5945-41f3-8189-6d8f7bbc503f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.733003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.771052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.771935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.772267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.772549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] PATH_INFO: `/v3/users/0bbd0af1acc54978824d3fe702dd202a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.773465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 122/248] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:53:57.774079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] RBAC: Authorizing `identity:get_user(user_id=0bbd0af1acc54978824d3fe702dd202a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.775439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9fb9-47ef-4016-a521-2f3d71aa74e7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.775660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.776157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9fb9-47ef-4016-a521-2f3d71aa74e7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.776635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd9fb9-47ef-4016-a521-2f3d71aa74e7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.777621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 127/249] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/users/0bbd0af1acc54978824d3fe702dd202a => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:57.778318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 123/250] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:53:57.782882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.784182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9fb9-47ef-4016-a521-2f3d71aa74e7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:57.893693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc9112-36d4-4de0-a272-e57465de7edf None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.900814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd9fb9-47ef-4016-a521-2f3d71aa74e7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:57.924369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:57.930985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.932028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.932509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.932797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.934036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.935471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21069de-bb87-496a-a582-4932736b7576 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.941926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:57.941926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.941926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.941926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.957585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:57.958446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:57.980824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:57.982035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:57.982522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:57.983026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:57.985063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:57.986746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:57.988693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 124/252] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:57.989649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.040837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.045019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 129/253] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:53:58.045667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.128028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedcf33-6079-4757-9f2b-d9852e1ca24a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.163722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.164894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.165384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.165874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.166907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=485e5609c19044d3a978abceedcffeb1) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:58.167420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:58.169207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:58.176376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 125/254] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:57 2022] GET /identity/v3/role_assignments?user.id=485e5609c19044d3a978abceedcffeb1&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:58.177468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.195488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f22c3-8ea7-4ac1-a047-07ac60127dad None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.236516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.242310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.243212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.243497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.243773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/0bbd0af1acc54978824d3fe702dd202a/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.245813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, user_id=0bbd0af1acc54978824d3fe702dd202a, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:58.247268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258bac6-b7b5-40e8-a6d1-48a8682aa9d1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:58.264559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 130/255] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:53:58 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/0bbd0af1acc54978824d3fe702dd202a/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:53:58.268390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3248-36a0-4b87-ba9e-6fa8e375c050 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.268982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3248-36a0-4b87-ba9e-6fa8e375c050 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.269458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd3248-36a0-4b87-ba9e-6fa8e375c050 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.271427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 131/256] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:53:58.277515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.278179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.278658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.302561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:53:58.306400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:53:58.372631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acc604-aa35-4892-9dd9-778d6dc09d7d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.380141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.411724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 132/257] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:53:58.435485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.438193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.439687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.441283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.441743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.455472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:58.467795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:58.468848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:53:58.469641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:53:58.472675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 126/258] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:58.478974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.574303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57579fc9-eba9-405a-8a59-d59f736ac1ac None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.593537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.603741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c47cc-3f51-47aa-897a-3e2014e7232a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.612425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.613490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.613932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.614350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.636270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:53:58.645594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:53:58.646714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:53:58.647719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:53:58.649252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.649999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.650267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 133/259] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:53:58.650366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.650592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.651326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:58.651640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:58.655295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:58.656405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.662127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 127/260] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:58.689239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.756485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.771642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288e7c6-8e77-4bcf-a348-6cdad569e885 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.832828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.833838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.834303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.834755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.835702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:58.836226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:58.837889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:58.847261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 134/261] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:58.852975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:58.872139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3425336d-b254-4fec-874d-cd7c1fc0e5f9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:58.918772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:58.947810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:58.948648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:58.948938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:58.949219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:58.950816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:58.952310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:58.954573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 128/262] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:58.961066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:59.023160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd974946-500f-4884-9ac5-110a9d48576f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:59.088401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:59.089660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:59.090138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:59.090616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] PATH_INFO: `/v3/users/17be58d6d34d4c488c340743b85df57e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:59.094653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] RBAC: Authorizing `identity:get_user(user_id=17be58d6d34d4c488c340743b85df57e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:59.096334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:59.100892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 135/263] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/users/17be58d6d34d4c488c340743b85df57e => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:59.106226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:59.133175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:59.149386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fec76c-6563-4436-9f54-281dbb630925 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:59.228398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:59.228973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:59.229265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:59.229551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:59.334900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef93a-8c8f-452e-8cdb-f304881c6981 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:59.360506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:59.365771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:59.367261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:59.421407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:59.423009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:59.423540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:59.424032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:59.479373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:59.491244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 129/264] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:53:59.499009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:59.560266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:59.562197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:59.644160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbac27a-60f3-414f-88da-1a258e94c439 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:59.684257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:59.685161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:59.685658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:59.685944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:59.688247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:59.689724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:59.691748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 130/265] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:59 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:53:59.700132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:59.761733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:59.815939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 136/266] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:53:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:53:59.821769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:53:59.824120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f5fcb-5238-437e-ae62-78f81fd64f1c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:53:59.844303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:53:59.894391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:53:59.895049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:53:59.895330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:53:59.895599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:53:59.896460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=17be58d6d34d4c488c340743b85df57e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:53:59.896768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:53:59.899286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:53:59.911025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 131/267] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:53:59 2022] GET /identity/v3/role_assignments?user.id=17be58d6d34d4c488c340743b85df57e&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:53:59.976378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e03884-2d71-4e06-811b-bf73839f3aec None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:00.022192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:00.022974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.023256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.023527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.024751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:00.026218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f68bc-8c4d-4768-9865-a34afa14ab27 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:00.028073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 137/268] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:53:59 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:00.034651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:00.144612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33dcebb-c08b-4bc0-8f31-0ac6a93ce548 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:00.169248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:00.182575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:00.183285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.183573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.183851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.184738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=17be58d6d34d4c488c340743b85df57e) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:00.185051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:00.186506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae508f-2cc6-4ba0-999b-68e81b6978fa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:00.194024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 132/269] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:00 2022] GET /identity/v3/role_assignments?user.id=17be58d6d34d4c488c340743b85df57e&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:00.482482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ed27-901a-463d-88f4-c7ecac450d26 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.483136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ed27-901a-463d-88f4-c7ecac450d26 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.483136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380ed27-901a-463d-88f4-c7ecac450d26 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.484923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 138/270] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:00.492194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b6b48-1eba-43e6-b972-586ae4a80b31 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.492973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b6b48-1eba-43e6-b972-586ae4a80b31 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.493412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b6b48-1eba-43e6-b972-586ae4a80b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.517223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bc9b6b48-1eba-43e6-b972-586ae4a80b31 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:00.518657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bc9b6b48-1eba-43e6-b972-586ae4a80b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:00.600446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 133/271] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:00.623877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380ed27-901a-463d-88f4-c7ecac450d26 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:00.691960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df620b-08ef-4738-959e-3ce4282dc477 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.692320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df620b-08ef-4738-959e-3ce4282dc477 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.692775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df620b-08ef-4738-959e-3ce4282dc477 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.694151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 134/272] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:00.699513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.700057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.700550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.722630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:00.725094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:00.726708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:00.748415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380ed27-901a-463d-88f4-c7ecac450d26 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:00.785097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:00.786716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.787472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.787946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.801672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 135/273] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:00.812787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:00.821093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:00.821983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:00.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:00.825509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 139/274] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:00.830851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:00.831928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:00.877835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:00.942875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6e54-8f55-4b90-bdf2-a8b44d792c0d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:00.950386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ec795-38b3-4d4e-87a6-2ac18b406595 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:00.986804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:00.987602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.987893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.988204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.989151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:00.989470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:00.991183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:00.992095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:00.993913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:00.994599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:00.995170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:00.995648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:00.997178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:00.999426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.001988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 136/275] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:01.004872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 140/276] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:01.009290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.010764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.140303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1e391-05ed-477b-bc68-981f2cc97034 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.141223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5efa2a-1b4f-4d0d-aff0-d065f6967b52 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.160206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:01.181548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.182415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.182558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.182762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.183075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] PATH_INFO: `/v3/users/d7289a5d5aa846658f736bbba7938d70` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.183168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.183431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.183709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.184496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:01.185950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b152d3-b29c-4a91-9afb-a0ac8e006620 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.198244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] RBAC: Authorizing `identity:get_user(user_id=d7289a5d5aa846658f736bbba7938d70)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:01.199824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bddf8c7-4e65-4b97-a487-107d0c234a2f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.202075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 141/277] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/users/d7289a5d5aa846658f736bbba7938d70 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:01.203336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa077a8-6049-4abc-a98e-0f8892801461 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.203611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa077a8-6049-4abc-a98e-0f8892801461 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.203884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa077a8-6049-4abc-a98e-0f8892801461 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.208550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 142/278] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:01.209286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa077a8-6049-4abc-a98e-0f8892801461 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.227432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 137/279] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:01 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:01.227432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.227432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.227432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.244300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:01.246247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:01.292396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:01.347895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 138/280] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:01.363900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa077a8-6049-4abc-a98e-0f8892801461 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.371073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.405651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.406404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.406644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.406922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.408160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, user.id=d7289a5d5aa846658f736bbba7938d70, scope.system=all) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:01.408160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:01.410641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c2e1e-95d1-4c39-8505-25e720b1ccbc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.414772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 143/281] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/role_assignments?user.id=d7289a5d5aa846658f736bbba7938d70&scope.system=all&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:01.416076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d6f1-415d-40a0-a8d4-e2ac6b696c27 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.416898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d6f1-415d-40a0-a8d4-e2ac6b696c27 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.417181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469d6f1-415d-40a0-a8d4-e2ac6b696c27 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.418601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 144/282] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:01.424394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.426633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.426633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.449512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:01.450848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:01.507078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455a22d-8829-4cb2-adc2-b9654599b9f7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.508904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:01.551420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 145/283] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:01.555903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.556836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.557215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.557569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.572181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:01.582287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:01.583225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:01.584390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:01.585278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 139/284] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:01.586506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.592572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.652191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:01.715362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84757-b269-40ee-a465-ff2fd170cab1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.718591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbab41-1dc1-428c-ae5f-45e8a9134de1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.764649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.765752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.766213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.766480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.767516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:01.767829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:01.769277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.779725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.780560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.780909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.781187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.790761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 140/285] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:01.805520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:01.813911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:01.814777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:01.815593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:01.820418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 146/286] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:01.821649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.822785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:01.858512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:01.930550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c7daf-3ab0-401e-b364-a4937a854b90 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:01.970090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:01.971104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:01.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:01.972992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:01.974801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:01.976393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:01.978014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:01.997846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 141/287] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:02.004484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:02.022071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac8da5-81c0-46bd-a397-0615fe7ea02d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.038763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:02.065043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.066222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.066699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.067191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] PATH_INFO: `/v3/users/485e5609c19044d3a978abceedcffeb1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.068606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] RBAC: Authorizing `identity:get_user(user_id=485e5609c19044d3a978abceedcffeb1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.070306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.072542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 147/288] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:01 2022] GET /identity/v3/users/485e5609c19044d3a978abceedcffeb1 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:02.081199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:02.112671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:02.132133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c596d7-b6ef-49ef-84c0-48c3efa8f49b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.166160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.167087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.167593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.167879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] PATH_INFO: `/v3/users/0bbd0af1acc54978824d3fe702dd202a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.171047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbd0af1acc54978824d3fe702dd202a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.173471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.175182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 142/289] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:02 2022] GET /identity/v3/users/0bbd0af1acc54978824d3fe702dd202a => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:02.180826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:02.231250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276be5ce-cc35-4432-8b1a-462119168a84 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.269776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.270643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.270929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.271209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.272511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.274741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.276497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 148/290] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:02 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:02.282503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:02.335850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:02.399850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf78b7-8929-4b90-b462-418ea777293a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.437873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.439320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.439746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.440184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/485e5609c19044d3a978abceedcffeb1/roles/e7f74e1aab7b4bd79af410095f5921a2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.442917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, user_id=485e5609c19044d3a978abceedcffeb1, role_id=e7f74e1aab7b4bd79af410095f5921a2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.444724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.460208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 149/291] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:02 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/users/485e5609c19044d3a978abceedcffeb1/roles/e7f74e1aab7b4bd79af410095f5921a2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:02.479777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7857a-5e8e-461a-b2ca-7e3539f4c849 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.523383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.524075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.524347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.524628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.555409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:02.614165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.615599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.721487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 143/292] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:02.727133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:02.904936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97b8dd-af3e-4379-b4c3-94ebd498e001 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:02.919702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:02.958543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:02.959407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:02.959696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:02.959976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:02.964831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:02.966304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436fe364-e0e4-4056-a5e7-60ced4d40d60 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:02.968365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 150/293] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:02 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:02.973813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:03.056040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:03.106166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29206606-f1ce-4430-93a6-4a69b7fd345d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:03.147117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:03.148015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:03.148477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:03.148894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:03.149866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=0bbd0af1acc54978824d3fe702dd202a) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:03.150338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:03.151917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6edd56-b3c6-461a-882a-ddd03c8a7c21 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:03.165176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 144/294] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:02 2022] GET /identity/v3/role_assignments?user.id=0bbd0af1acc54978824d3fe702dd202a&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:03.734378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c36a-ca76-458e-8bab-ce8f9e1214d1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:03.734892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c36a-ca76-458e-8bab-ce8f9e1214d1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:03.735174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270c36a-ca76-458e-8bab-ce8f9e1214d1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:03.736960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 151/295] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:03.742162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16cdc9-a305-4a1b-a1a8-8ca4abf230e7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:03.742558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16cdc9-a305-4a1b-a1a8-8ca4abf230e7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:03.742920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16cdc9-a305-4a1b-a1a8-8ca4abf230e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:03.763409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bd16cdc9-a305-4a1b-a1a8-8ca4abf230e7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:03.764651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bd16cdc9-a305-4a1b-a1a8-8ca4abf230e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:03.848537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 145/296] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:03.856936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36459-323e-482e-a39a-86cdabdbbd61 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:03.857226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36459-323e-482e-a39a-86cdabdbbd61 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:03.857507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36459-323e-482e-a39a-86cdabdbbd61 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:03.859637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 152/297] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:03.865461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:03.866035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:03.866505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:03.872257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36459-323e-482e-a39a-86cdabdbbd61 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:03.889951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:03.892053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:03.956394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:03.988291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 146/298] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:03.992797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36459-323e-482e-a39a-86cdabdbbd61 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.010178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.034535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.035641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.036161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.036871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.048710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f356b8b8-c8af-4547-9f94-9d834c021d7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:04.056694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:04.057734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:04.058721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 153/299] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:04.061907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7830-3f45-4cbc-b23a-4626dda89923 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.062270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7830-3f45-4cbc-b23a-4626dda89923 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.062710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f7830-3f45-4cbc-b23a-4626dda89923 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.064438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 154/300] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:04.068928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7830-3f45-4cbc-b23a-4626dda89923 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.104366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.126849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ad718-2bfa-4777-8833-fe6973f45a06 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.162415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.163416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.163838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.164286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.171921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f7830-3f45-4cbc-b23a-4626dda89923 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.176017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-82e985f9-0cf0-482c-bb73-5c054ee9de45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:04.185325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:04.186452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:04.187466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:04.189414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 147/301] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:04.190598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.190894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.191175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.211544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:04.212749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:04.213041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.213885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.214347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.214909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.216583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:04.217089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.219250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.228605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 155/302] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.229800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.246695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.285112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 148/303] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:04.289024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.342108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.348283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325dfb9-8b83-4e37-a8fe-c4dbbde2a2dd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.383375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.384036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.384349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.384635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.385388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:04.386125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.387521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.396506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 156/304] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.397289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.403832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3570b-9a5f-41fd-b3e7-53cff6ced3e8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.441357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.442229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.442510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.442808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] PATH_INFO: `/v3/users/57d7952bd79646bf9dd58af216fd1595` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.457190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] RBAC: Authorizing `identity:get_user(user_id=57d7952bd79646bf9dd58af216fd1595)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.458650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.464838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 149/305] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/users/57d7952bd79646bf9dd58af216fd1595 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.464838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.491574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.511322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103e397-862f-495f-9e2e-28cc68a449a7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.549194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.549987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.550273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.550823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.551714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.553187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.559133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 157/306] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.559133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.572990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ece31a-dc62-49e5-b4a4-9f2b34c00038 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.588558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.612988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.613881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.614168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.614441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] PATH_INFO: `/v3/users/d7289a5d5aa846658f736bbba7938d70` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.615657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] RBAC: Authorizing `identity:get_user(user_id=d7289a5d5aa846658f736bbba7938d70)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.617154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.618947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 150/307] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/users/d7289a5d5aa846658f736bbba7938d70 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.621479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.673929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826e013-9627-4bb1-8ae1-8b787ffed360 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.717308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.717970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.718255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.718531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.719362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, user.id=57d7952bd79646bf9dd58af216fd1595, scope.system=all) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:04.719673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.721145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.736156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 158/308] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Jun 20 06:54:04 2022] GET /identity/v3/role_assignments?user.id=57d7952bd79646bf9dd58af216fd1595&scope.system=all&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:04.736782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.736872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:04.747906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aafbea-75f2-406b-bd35-f0c03a5a106f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.802560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.803877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.804504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.804789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] PATH_INFO: `/v3/groups` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.805605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.807003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614e4d2-ac70-40cc-a90b-0365eecff22b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.832651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 151/309] 10.222.0.23 () {60 vars in 1134 bytes} [Mon Jun 20 06:54:04 2022] POST /identity/v3/groups => generated 229 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:04.842567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfcb31-a993-43c6-bced-0ea6d63cfea6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.842799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfcb31-a993-43c6-bced-0ea6d63cfea6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.843034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdfcb31-a993-43c6-bced-0ea6d63cfea6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.844815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 152/310] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:04.852134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:04.852347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.852347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.852347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.865024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fdb8e-c2b8-433a-a2fc-c07255a86f6b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:04.870472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:04.871936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:04.918113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:04.919217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:04.919503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:04.919895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] PATH_INFO: `/v3/system/users/d7289a5d5aa846658f736bbba7938d70/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:04.922112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d7289a5d5aa846658f736bbba7938d70, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:04.923523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:04.940764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 159/311] 10.222.0.23 () {58 vars in 1334 bytes} [Mon Jun 20 06:54:04 2022] PUT /identity/v3/system/users/d7289a5d5aa846658f736bbba7938d70/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:04.976567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 153/312] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:05.007861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:05.127123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41aca99-45cb-4b29-97e4-4d0ece7ce4d4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:05.162837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:05.163639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:05.163929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:05.164242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:05.174885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:05.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:05.183552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:05.184434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 160/313] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:05.190257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:05.279093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:05.297026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6ad65-f467-468b-9ea9-d4ad03b688cb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:05.337222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:05.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:05.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:05.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:05.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:05.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:05.344138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:05.349226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 154/314] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:05.354717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:05.372364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:05.463918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937a436-fcc6-4655-a112-0914013ed5c4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:05.498338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:05.499228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:05.500041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:05.500350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] PATH_INFO: `/v3/users/485e5609c19044d3a978abceedcffeb1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:05.501561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] RBAC: Authorizing `identity:get_user(user_id=485e5609c19044d3a978abceedcffeb1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:05.503015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:05.505023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 161/315] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/users/485e5609c19044d3a978abceedcffeb1 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:05.510411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:05.618032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f7379-c3f5-42f1-93f5-5b2057b7f4e1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:05.653679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:05.654738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:05.655267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:05.655772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:05.672749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:05.729172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:05.730565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:05.831954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 155/316] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:05.837939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:05.928749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:05.947551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f076dd4-f55c-4d7f-8d93-e801f458c80d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:05.982048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:05.982952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:05.983235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:05.983530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:05.985323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:05.986831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e702d-ef72-4ede-a5a3-7902dc72245f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:05.988889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 162/317] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:05.997195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:06.060477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:06.106020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48688f69-3b5a-482c-ad1e-f03916512854 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:06.140888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:06.141545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:06.142082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:06.142359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:06.143191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, scope.project.id=379a41eb23be41efa1ea7937bcde4175, user.id=485e5609c19044d3a978abceedcffeb1) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:06.143503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:06.144988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36112d-790f-4003-8812-b5a04e95c211 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:06.151218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 156/318] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:05 2022] GET /identity/v3/role_assignments?user.id=485e5609c19044d3a978abceedcffeb1&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:06.845059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5eb4a-7c8e-4cab-be9e-8adab18013ad None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:06.845555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5eb4a-7c8e-4cab-be9e-8adab18013ad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:06.845627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5eb4a-7c8e-4cab-be9e-8adab18013ad None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:06.846991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 163/319] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:06.852307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:06.852675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:06.852876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:06.870107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:06.871320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:06.935140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e637-f118-4121-8f65-3138cb2deb06 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:06.937629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e637-f118-4121-8f65-3138cb2deb06 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:06.937629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e637-f118-4121-8f65-3138cb2deb06 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:06.937629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 164/320] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:06.942637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:06.942915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:06.943190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:06.944850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 157/321] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:06.960086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:06.961234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:06.967146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.013668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.040132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 165/322] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:07.053781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.080440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a76c09-146a-499f-890c-4ac69785f45a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.116234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.117293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.117584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.117864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.128219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-80884d84-2778-430f-b2b7-eee34d363c03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:07.135799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:07.136889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:07.137865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:07.138575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.138659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 158/323] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:07.138659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d6c2-b847-45bb-9cc2-5bc15cff0127 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.138968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d6c2-b847-45bb-9cc2-5bc15cff0127 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.139245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d6c2-b847-45bb-9cc2-5bc15cff0127 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.141060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 159/324] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:07.143895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d6c2-b847-45bb-9cc2-5bc15cff0127 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.159126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00553952-6d25-4755-aa21-71da5b2cb508 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.192340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.193130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.193410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.193689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.194853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.196328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504888dd-dcca-42c4-893f-ff5c7d255254 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.197950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 166/325] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:07.200639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.200639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.200639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.217487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:07.219228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:07.244791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.249615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d6c2-b847-45bb-9cc2-5bc15cff0127 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.282838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.283497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.283786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.284068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.284674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 167/326] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:07.284858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:07.285158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.285474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.286553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.295970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 160/327] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:07.301122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.362047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.384201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faf837-d72d-4581-9218-a4428c8b3452 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.398461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b791-fffd-43e8-b87e-5f5af1ccc363 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.418165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.418860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.419153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.419430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] PATH_INFO: `/v3/groups` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.420146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.421637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.433969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.434827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.435105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.435376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] PATH_INFO: `/v3/users/57d7952bd79646bf9dd58af216fd1595` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.437296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] RBAC: Authorizing `identity:get_user(user_id=57d7952bd79646bf9dd58af216fd1595)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.437921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 168/328] 10.222.0.23 () {60 vars in 1134 bytes} [Mon Jun 20 06:54:07 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:07.438751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.439157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.441138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 161/329] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/users/57d7952bd79646bf9dd58af216fd1595 => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:07.454559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.489122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.561023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3cc1e-f73e-447b-9780-4c8f219f8005 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.568381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128c4b1-7205-40f4-8e57-c8ffb34696f2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.601102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.602160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.602598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.602876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.611236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.611752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.612031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.612392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] PATH_INFO: `/v3/system/users/57d7952bd79646bf9dd58af216fd1595/roles/e7f74e1aab7b4bd79af410095f5921a2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.614008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=57d7952bd79646bf9dd58af216fd1595, role_id=e7f74e1aab7b4bd79af410095f5921a2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.616816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.616942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.624136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:07.627963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 162/330] 10.222.0.23 () {58 vars in 1334 bytes} [Mon Jun 20 06:54:07 2022] PUT /identity/v3/system/users/57d7952bd79646bf9dd58af216fd1595/roles/e7f74e1aab7b4bd79af410095f5921a2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:07.633697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:07.634855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:07.635849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 169/331] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:07.641247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.756917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:07.765971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2158ca-2115-4d0a-8cdd-3bb03e3eead1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.810282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.811111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.811470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.812406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.813165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:07.813473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.814858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.823976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 163/332] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:07.829391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:07.931644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc174-a28e-4155-925d-54a524c68095 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:07.969320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:07.970201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:07.970480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:07.970751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] PATH_INFO: `/v3/users/d7289a5d5aa846658f736bbba7938d70` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:07.971977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] RBAC: Authorizing `identity:get_user(user_id=d7289a5d5aa846658f736bbba7938d70)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:07.973490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1be7368-9aec-4a87-85d2-882ec5adde0b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:07.976186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 170/333] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/users/d7289a5d5aa846658f736bbba7938d70 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:07.981410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:08.117716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f02872-9000-4fab-9fa6-d867771ebc75 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:08.156136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:08.156811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:08.157095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:08.157373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:08.158215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] RBAC: Adding query filter params (role.id=b1643f7fcdff4280b6a98fb292c0e2cb, user.id=d7289a5d5aa846658f736bbba7938d70, scope.system=all) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:08.158525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:08.159972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8e2547-e624-47de-97b2-285d414680e0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:08.173276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 164/334] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Jun 20 06:54:07 2022] GET /identity/v3/role_assignments?user.id=d7289a5d5aa846658f736bbba7938d70&scope.system=all&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:09.798638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bca26-92f0-47d1-b279-5f733e45a6c9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:09.798638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bca26-92f0-47d1-b279-5f733e45a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:09.798638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095bca26-92f0-47d1-b279-5f733e45a6c9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:09.798638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 171/335] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:09.804236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:09.804780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:09.804780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:09.834987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:09.836219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:09.848142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415fc2-5363-40c4-a8d1-000186eca31c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:09.848142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415fc2-5363-40c4-a8d1-000186eca31c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:09.848142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39415fc2-5363-40c4-a8d1-000186eca31c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:09.849855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 172/336] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:09.851192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:09.857489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:09.858055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:09.859122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:09.893224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:09.894761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:09.925737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 165/337] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:09.973807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:09.984476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:09.995239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 173/338] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:10.037735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.101079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e180f-e8a0-402b-8c02-dcfab8705dff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.145766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.146573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.146854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.147155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.158874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:10.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:10.173377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:10.174202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 166/339] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:10.179375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.183673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:10.187282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1751b82-7315-4a2f-ac95-df7ac62aec9f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.243621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.244812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.245296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.245764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.261558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:10.270308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:10.271190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 06:54:10.271992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:10.278867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 174/340] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:10.285364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.329959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45845c3-117d-4fa1-8902-8e1e8544c435 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.364001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:10.380141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.380141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.380397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.380501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.381484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:10.381841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:10.383239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:10.399081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 167/341] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:10.399081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.448281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c013c2-4441-4264-8bea-952ab23ffd1b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.523678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:10.531779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.532480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.532765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.533040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.533802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:10.534107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:10.535542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:10.556677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 175/342] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:10.573463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897beffa-eea5-4e23-ab8b-76005e3cd614 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.585662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.617880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.619221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.619782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.620421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.703762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:10.705632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:10.783580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:10.813285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4c6be-6fef-46ba-9099-8aa0c97c0f56 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:10.827850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 168/343] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:54:10.834714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:10.849282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:10.888851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:10.890269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.890552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.890824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] PATH_INFO: `/v3/users/57d7952bd79646bf9dd58af216fd1595` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.892406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] RBAC: Authorizing `identity:get_user(user_id=57d7952bd79646bf9dd58af216fd1595)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:10.893821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760569b8-e2ea-4d2f-ba5c-0a7f70dd9835 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:10.900051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 176/344] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/users/57d7952bd79646bf9dd58af216fd1595 => generated 293 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:10.900051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72568283-0400-4abf-8b3b-273c5c9617a0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.900051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72568283-0400-4abf-8b3b-273c5c9617a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.900051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72568283-0400-4abf-8b3b-273c5c9617a0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.900403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 177/345] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:10.910507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:10.910778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:10.911044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:10.948613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:10.949804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:10.965636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327db4aa-9689-4799-a90c-2a4b84c75c98 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.010307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.011584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.013299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.013787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.017763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:11.019519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:11.023069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 169/346] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:10 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:11.024073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.070255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:11.090171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 178/347] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:11.091095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.160267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ead7bd-a6c3-457d-b740-c11839298cdb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.201250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.201929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.202797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.203070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.204007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] RBAC: Adding query filter params (role.id=e7f74e1aab7b4bd79af410095f5921a2, user.id=57d7952bd79646bf9dd58af216fd1595, scope.system=all) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:11.204379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:11.205987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954ccc6-d67a-4c3a-9011-b0bafd40dbf3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:11.212778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3843a-eb37-4b41-969b-14749c43bd9c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.219685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 170/348] 10.222.0.23 () {56 vars in 1308 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/role_assignments?user.id=57d7952bd79646bf9dd58af216fd1595&scope.system=all&role.id=e7f74e1aab7b4bd79af410095f5921a2 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:11.221962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791476b-5046-46ad-b6bb-01cceb502722 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.221962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791476b-5046-46ad-b6bb-01cceb502722 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.221962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791476b-5046-46ad-b6bb-01cceb502722 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 171/349] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:11.227368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791476b-5046-46ad-b6bb-01cceb502722 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.248640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:11.288604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.289519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.289805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.290078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.298739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:11.300504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2123100-2e4d-4abc-b30d-80492fb07bfa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:11.302282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 179/350] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:11.303654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.303943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.308133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.326987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:11.328858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:11.431425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:11.445402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791476b-5046-46ad-b6bb-01cceb502722 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.475283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 180/351] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:11.476609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.532058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.534013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.534515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.535276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.555649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:11.564295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:11.565266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:11.566110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 172/352] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:11.566787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.617223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef05bd-9dc2-445f-a521-6052c9dccbbd None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.631416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:11.678517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.679639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.679922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.680620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.682349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=15f6d807458c48ecb1d58725592095f0) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:11.682669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:11.684400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:11.699359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 181/353] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:11.700704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.718205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d20-22ec-4320-81ad-c505dca5d183 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.799317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.801803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.802090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.802367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.831042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:11.843420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:11.844416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:11.845251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 173/354] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:11.852342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:11.878906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:11.885236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1241ea-6afb-4aa1-95b6-592d369ee0d9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:11.930927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:11.931897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.932933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.933418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.934350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:11.934940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:11.937371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3bb4a-fc8d-4f41-ab77-eaa9c0755ea2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:11.947948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 182/355] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:11.949468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5a8c-8692-4fcf-9a0c-85f293ef2604 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.949963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5a8c-8692-4fcf-9a0c-85f293ef2604 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.950415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c5a8c-8692-4fcf-9a0c-85f293ef2604 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.952169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 183/356] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:11.960157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:11.960157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:11.960157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:11.977709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:11.979117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:11.979233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e44e2-37a4-4f8c-86ed-a170d51203ad None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.010732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.018316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.019336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.019831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.020319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.021295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:12.021793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.023345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.033767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 174/357] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:12.034856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.058866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 184/358] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:12.059981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.151167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.179677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe42d33-f0e5-4e90-a87b-310cbfeffdbd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.187050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0115a9-012a-4213-8c38-1f89a76da609 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.224470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.225129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.225414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.225692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.231595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.232277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.232559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.232833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.279532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.310612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.312852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.321049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.322506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3174959f-e75f-41d1-934d-9a9aeb318007 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.410307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.439754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 175/359] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:12.440751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.459391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 185/360] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:12.461331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5712-8e3a-45c5-8fa4-e33311e42a3c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.461629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5712-8e3a-45c5-8fa4-e33311e42a3c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.461911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5712-8e3a-45c5-8fa4-e33311e42a3c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.463325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 186/361] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:12.464065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5712-8e3a-45c5-8fa4-e33311e42a3c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.547562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.594181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98080cb-cf49-4fa9-ba3d-759be627a9eb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.598340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5712-8e3a-45c5-8fa4-e33311e42a3c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.638770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.639857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.640879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.640879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.641079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.641917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.642200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.642470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.645169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.646617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.648971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 187/362] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:12.650003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.653164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f9724a6a-2c38-4f58-90c8-3be2977b4645 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:12.663430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:12.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:12.665351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:12.666008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 176/363] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:12.667074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.667504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.667668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.687877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:12.689548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:12.722015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.768799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 177/364] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:12.769740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.789040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45879bb-7476-4406-ab98-a5e234b9fda2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.840141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.841317 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.841601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.841884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.844257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.845863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.847567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 188/365] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:12.848643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.851139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:12.907674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57442e-94cb-4bf5-aeeb-3739afec6b6c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.947975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:12.948813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:12.949098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:12.949378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:12.950560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:12.952573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:12.954389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 178/366] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:12.954969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:12.977513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8594511-5044-4c80-ace3-78caf4cf89c4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:12.999962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.020720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.022052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.022511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.022789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.023548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:13.023850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.027837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.034748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 189/367] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:13.035382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.088653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb007c8-e2bc-4191-aa26-3453a2ef2016 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.116132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.136393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.137194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.137535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.137757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.152344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:13.160529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:13.161406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:13.162243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 179/368] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:13.162907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.167507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcb0479-f157-4800-8416-0a358068a21a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.211621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.213081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.213372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.214334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.214870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.218015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.224298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 190/369] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:13.224298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.292140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.304847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f28089-5842-4b56-8a1d-1561469a3fb2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.343283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.343942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.344259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.344539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.345376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=379a41eb23be41efa1ea7937bcde4175) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:13.345883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.347125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.355761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fba3c-3139-45d2-94b1-eea715fc7eb3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.355929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 180/370] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:13.355929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.403007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.413425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.415208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.415728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.416279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.513852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.515915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.516760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ac4a3-8ef6-468f-84d2-d363fc76fa48 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.571706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.572623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.572912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.573579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.574618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:13.574934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.576517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.592473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 181/371] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:13.593533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.617041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.644150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 191/372] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:13.644150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.681342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.737639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aed4cd-d23f-4e1e-a6cc-3b6b40ba78c6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.762157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5189b3d-1f61-4da4-a131-b8c88c065c20 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:13.786490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.787520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.787949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.788388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.789381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=15f6d807458c48ecb1d58725592095f0) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:13.789858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.791303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.802805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 182/373] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:13.802805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:13.805607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:13.806467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:13.806901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:13.807333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:13.905538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:13.907827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5c1cc-5101-4fe0-80e4-f6da50c63ac3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:13.957734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:13.976693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.015028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e441eb-8872-4986-8b54-9b42afb97d30 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.064438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 192/374] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:14.066008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965afe0-7692-4f4e-83ec-0d8cb05c4fcc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.066385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965afe0-7692-4f4e-83ec-0d8cb05c4fcc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.066676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1965afe0-7692-4f4e-83ec-0d8cb05c4fcc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.066954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.067818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.068099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.068333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 193/375] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:14.068976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965afe0-7692-4f4e-83ec-0d8cb05c4fcc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.072139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.075194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.076750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d662dc0-2833-45ed-aa9d-ed7bb638d11e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.079024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 183/376] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:13 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.080258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.080545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.080831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.105150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:14.106512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:14.199027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 184/377] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:14.200401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.208526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1965afe0-7692-4f4e-83ec-0d8cb05c4fcc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.245146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.247687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.249304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.249579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.249847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] PATH_INFO: `/v3/projects/649b350e7f364263b37fbd34d0f07b2b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.251553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] RBAC: Authorizing `identity:get_project(project_id=649b350e7f364263b37fbd34d0f07b2b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.254659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.259882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 194/378] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/projects/649b350e7f364263b37fbd34d0f07b2b => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.259882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.300181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.323232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a3286-63cd-4a2b-9a2c-cb113247eb0d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.368303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.369173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.369463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.369745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.371491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.373262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.376311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 185/379] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.377373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.389515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584ab5b-20c2-4a6c-b804-f3ff6bb3b374 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.429747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.430887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.431369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.431864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.444398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:14.453208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:14.454181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:14.454956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:14.457309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 195/380] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:14.458449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.515529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a05d2-185a-4778-9f95-9be28b3b73ac None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.549274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.556451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.556892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.557173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.557448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] PATH_INFO: `/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.567247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] RBAC: Authorizing `identity:get_group(group_id=1d29ccfbde554fa7922c186a2fc0a24f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.568822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.574734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 186/381] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.575597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.594242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74725b8-e13e-4b9b-b27a-b52552f17f08 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.596908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.635635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.636419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.636757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.637042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.637881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=379a41eb23be41efa1ea7937bcde4175) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:14.638195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.639654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.648786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 196/382] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.649630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.745346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a21889-5337-44ca-bb08-4a5c16aad411 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.803277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ffee4-883b-473d-a2fc-d5b8529447e1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:14.813752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.814434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.814718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.814993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.815854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:14.816190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.817680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.833053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.836668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 187/383] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.842908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:14.852785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:14.853463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:14.853753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:14.854034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:14.854884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] RBAC: Adding query filter params (group.id=1d29ccfbde554fa7922c186a2fc0a24f, role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=649b350e7f364263b37fbd34d0f07b2b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:14.855202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:14.856702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:14.872269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 197/384] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/role_assignments?group.id=1d29ccfbde554fa7922c186a2fc0a24f&scope.project.id=649b350e7f364263b37fbd34d0f07b2b&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:14.961815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:14.995019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3bcee-1eb9-4815-a79a-ea3147d5e6d0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:15.043656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:15.044615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:15.044905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:15.045190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:15.051526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:15.053049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:15.055139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 188/385] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:14 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:15.060703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:15.183045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05fb39-92af-4f67-9ec4-cc4998f655c1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:15.221926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:15.223049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:15.223345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:15.223630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:15.229363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:15.231562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b5fd5-625d-4fb1-abfb-6fd53922bc3c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:15.234526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 198/386] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:15 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:15.239983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:15.368649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bb10b-8ff4-4ee4-808f-6721b8f85efc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:15.412816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:15.413709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:15.414001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:15.414282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:15.416346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, group_id=b0a4d929c01d4ddfa35c61077c0f5517, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:15.417806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028304-7cfc-4621-acb6-79dfb3cfe057 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:15.444406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 189/387] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Jun 20 06:54:15 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:15.811842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3a4c5-231f-460a-be2a-be2cb39d52e7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:15.812382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3a4c5-231f-460a-be2a-be2cb39d52e7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:15.812820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a3a4c5-231f-460a-be2a-be2cb39d52e7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:15.814552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 199/388] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:15.821397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:15.821949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:15.822422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:15.829194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:15.843695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:15.845564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:15.933770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 190/389] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:15.968306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3a4c5-231f-460a-be2a-be2cb39d52e7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.038355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:16.087087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a3a4c5-231f-460a-be2a-be2cb39d52e7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.124466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.125255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.125540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.125816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.136362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-d8dacf4a-f5da-4b5c-9003-561de04d7b7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:16.144474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:16.145379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:16.146191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 200/390] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:16.151890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.197827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5c9cb-c118-4982-9b06-1001185bbadd None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.198441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5c9cb-c118-4982-9b06-1001185bbadd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.198441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5c9cb-c118-4982-9b06-1001185bbadd None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.199897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 201/391] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:16.205361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.205640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.205916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.227036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:16.228452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:16.314457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b464afaf-9767-43be-8f3b-8f306bd16acd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.315159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 202/392] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:16.356440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.365457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.368042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.368553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.370114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.371034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:16.372190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:16.373952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:16.377821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:16.396920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 191/393] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:16.399797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.447785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:16.556553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235586f-cc01-45fa-9d1b-bebfb7b5fcd7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.582944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b510f8-efa6-4512-8540-a0ed4e0fcbd8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.619498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.620708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.621165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.621610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.622995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:16.624707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:16.625346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.626158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.626443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.626719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.627609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 192/394] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:16.632955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.652834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:16.670880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:16.671786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:16.672628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 203/395] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:16.678334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.699329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:16.741727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b36e0-9f5f-46f5-a5d7-d8bf9612d412 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.775688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.776821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.777281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.777732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.779085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:16.781505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:16.783447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 193/396] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:16.789055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.841668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cdb225-3bda-4e21-ae8d-9ffc20c2f4c9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.850432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:16.902600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:16.903270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:16.903550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:16.903822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:16.906835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:16.907155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:16.908571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:16.923978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 204/397] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:16.931005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:16.969247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5706f9-f2e4-45e0-842e-8ad3280869f5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:16.999752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:17.014835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:17.015728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.016014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.016575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.019641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, group_id=b0a4d929c01d4ddfa35c61077c0f5517, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:17.021116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:17.047795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 194/398] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Jun 20 06:54:16 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:54:17.100004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebf46c-1f77-494a-9ed3-ed7c598491d9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:17.143199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:17.144080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.144458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.144722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.148994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:17.150486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:17.152279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 205/399] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:16 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:17.158203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:17.295324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa3cce-e78e-4e05-b183-72a00cb32db9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:17.331209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:17.332084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.334186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.334479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.335743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:17.337224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fd01b-934f-4fd9-bb5f-22b12b84d822 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:17.339236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 195/400] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:17 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:17.345181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:17.393539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:17.466711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e044-3fc4-4027-80c9-c09726f02215 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.467031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e044-3fc4-4027-80c9-c09726f02215 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.467299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546e044-3fc4-4027-80c9-c09726f02215 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.469079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 196/401] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:17.475580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b4f90-c4fc-426f-9b17-0d6f81ab7b23 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.475995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b4f90-c4fc-426f-9b17-0d6f81ab7b23 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.476330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b4f90-c4fc-426f-9b17-0d6f81ab7b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.497210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931938e7-b837-45ed-b597-57b82b76ede2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:17.497414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-563b4f90-c4fc-426f-9b17-0d6f81ab7b23 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:17.499453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-563b4f90-c4fc-426f-9b17-0d6f81ab7b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:17.568304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:17.569195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.569479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.569751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.574880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] RBAC: Authorizing `identity:create_grant(project_id=15f6d807458c48ecb1d58725592095f0, group_id=b0a4d929c01d4ddfa35c61077c0f5517, role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:17.576370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:17.598876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 206/402] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Jun 20 06:54:17 2022] PUT /identity/v3/projects/15f6d807458c48ecb1d58725592095f0/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:17.599981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:17.628798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 197/403] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:17.668882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:17.701786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe078f-cd44-454d-b584-dd9b1e3753b4 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.707254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe078f-cd44-454d-b584-dd9b1e3753b4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.707555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe078f-cd44-454d-b584-dd9b1e3753b4 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.709107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 198/404] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:17.716726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2690-828f-4322-861e-86038fe460db None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.717022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2690-828f-4322-861e-86038fe460db None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.717878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f2690-828f-4322-861e-86038fe460db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.742687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a08f2690-828f-4322-861e-86038fe460db None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:17.744012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a08f2690-828f-4322-861e-86038fe460db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:17.840240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 199/405] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:17.847998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:17.862235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eec808-b28e-42c3-ac0c-e8fdbf49eb47 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:17.865505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f2690-828f-4322-861e-86038fe460db None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:17.902067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:17.903570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:17.904033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:17.904506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:17.917670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:17.928146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:17.929043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:17.929854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:17.932804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 207/406] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:17.939852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.058577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f2690-828f-4322-861e-86038fe460db None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.095037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.114575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5395fd-577d-4cfa-8779-b2cbf60398fa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.130482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.131583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.132132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.132551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.144827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d3def073-944b-46b5-a39f-97f19211673a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:18.153368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:18.154281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:18.155164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:18.157635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 200/407] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:18.159042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0517f-596a-4419-bad2-2ca64c881a19 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.159593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.160027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0517f-596a-4419-bad2-2ca64c881a19 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.160465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0517f-596a-4419-bad2-2ca64c881a19 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.160591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.160934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.161392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.162256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:18.162772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.162920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 201/408] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:18.163740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0517f-596a-4419-bad2-2ca64c881a19 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.165851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449e53f-d793-4af5-ac72-e7655be3ae5d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.177693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 208/409] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:18.183394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.183394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.183394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.199586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:18.201833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:18.217610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.283712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0517f-596a-4419-bad2-2ca64c881a19 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.288506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 209/410] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:18.289435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.332962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.334112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.334408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.334694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.335468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:18.336219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.337951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.349854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 202/411] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.352098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.408382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.411940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0da2-b1e2-4c78-b4f2-cbdaa3d201ea None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.454554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.455333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.455614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.455887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.457122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.458566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.464172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 210/412] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.465218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.491258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.496378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83931c72-2a25-464d-9fc7-da0cd062be2c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.543738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.545634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.546274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.546843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.561686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:18.572634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:18.573607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:18.574428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:18.582506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 203/413] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:18.582506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.601118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54319d8-80e0-4f0a-95c1-3676b2f7dda1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.637580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.638780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.640279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.640770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] PATH_INFO: `/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.642884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] RBAC: Authorizing `identity:get_group(group_id=1d29ccfbde554fa7922c186a2fc0a24f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.645288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.652375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 211/414] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.653199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.705590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.710522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894dbd-3f29-40d8-9b13-ee0475f379ae None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.743720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:18.761512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e09f8-25d0-44f5-bead-15b87a56d761 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.761653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.762721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.763377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.763836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.768833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.770922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.773227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 204/415] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.774212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.803450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.804190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.804476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.804749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.805556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:18.805786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.807141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.820431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 212/416] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.820431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.894006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b518e337-7409-44f4-86df-f87a694e1b1d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.930542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:18.931363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:18.931648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:18.931930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] PATH_INFO: `/v3/projects/649b350e7f364263b37fbd34d0f07b2b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:18.933340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] RBAC: Authorizing `identity:get_project(project_id=649b350e7f364263b37fbd34d0f07b2b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:18.934830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:18.942021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 205/417] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/projects/649b350e7f364263b37fbd34d0f07b2b => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:18.942021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:18.961161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5129d-8bd4-4656-bd62-98bf413ce4e4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:18.981567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:19.006362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.007637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.008385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.008820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.012034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] RBAC: Authorizing `identity:create_grant(project_id=379a41eb23be41efa1ea7937bcde4175, group_id=b0a4d929c01d4ddfa35c61077c0f5517, role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.013992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.033084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 213/418] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Jun 20 06:54:18 2022] PUT /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175/groups/b0a4d929c01d4ddfa35c61077c0f5517/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:19.034528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:19.056146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71789dd-e8af-477f-8ea6-53378914849d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:19.056598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:19.130930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.131844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.132303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.132688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.162808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6beb4a-9904-4598-ba76-433a9681f21c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:19.204743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.205625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.205909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.206185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] PATH_INFO: `/v3/projects/649b350e7f364263b37fbd34d0f07b2b/groups/1d29ccfbde554fa7922c186a2fc0a24f/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.208162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] RBAC: Authorizing `identity:create_grant(project_id=649b350e7f364263b37fbd34d0f07b2b, group_id=1d29ccfbde554fa7922c186a2fc0a24f, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.209662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.227697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 214/419] 10.222.0.23 () {58 vars in 1442 bytes} [Mon Jun 20 06:54:19 2022] PUT /identity/v3/projects/649b350e7f364263b37fbd34d0f07b2b/groups/1d29ccfbde554fa7922c186a2fc0a24f/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:19.282227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.283886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.325395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:19.408223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 206/420] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:19.413852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:19.501398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:19.540276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670b952-0b72-4385-b54e-0a9336f7283a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:19.575893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.577746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.578271 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.578748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.580211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.582035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701de5a-50b5-46bc-b286-d1fc496448f5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.584292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 215/421] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:19 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:19.593448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:19.672625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5649e-6bca-4ec1-adf5-2a506d87694a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.672942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5649e-6bca-4ec1-adf5-2a506d87694a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.673180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5649e-6bca-4ec1-adf5-2a506d87694a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.674806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 216/422] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:19.679681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.679970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.680280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.723279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba045ee-8882-4bc0-9d70-fcba5bbd555d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:19.729141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:19.730350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:19.759174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.760038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.760467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.760762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.761942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.763390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.765442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 207/423] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:19 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:19.771355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:19.792486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:19.882081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16c246-405a-4c22-a040-728dc47d7866 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:19.919877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:19.921109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.921393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.921667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.922507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=15f6d807458c48ecb1d58725592095f0) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:19.922819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:19.924295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf07029c-478a-4e5e-85d5-f57f87c50472 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:19.936733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 208/424] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:19 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:19.938560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d524d0-1883-4cf7-a354-f9af54d59fbe None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.938856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d524d0-1883-4cf7-a354-f9af54d59fbe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.939136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d524d0-1883-4cf7-a354-f9af54d59fbe None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.940824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 209/425] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:19.946995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:19.947282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:19.947559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:19.948430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 217/426] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:19.972320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:19.981365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:19.982613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:20.059073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.087621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.096289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 210/427] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:20.134888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.150229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73781000-3c85-46a5-adbe-3efd19287ec9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.192577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.193663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.193944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.194216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.213177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:20.220894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:20.221991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:20.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 218/428] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:20.230962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.271385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b438394-e5bb-48cf-a9a2-1086dbdfcfc7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.327948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.328752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.332139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.332139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.347170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:20.358944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86897e50-0eeb-4869-92d1-fd01fa38106c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:20.360931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:20.363768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:20.365744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 211/429] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:20.373271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.386192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.407235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.407913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.408222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.408502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.409266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] RBAC: Adding query filter params (name=member) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:20.409573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:20.410965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:20.421168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 219/430] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:20.427479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.457741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.503143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04dfa1-fb17-47eb-9b30-6b66294de729 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.538416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.539080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.539365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.539645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.540442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:20.540750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:20.542121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:20.547797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c53190-030b-4744-8813-01e3438e19df None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.555287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 212/431] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:20.556726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.583841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.584809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.585319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.585786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.651580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.667278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44e886-3ada-4531-b4a4-7a85e0a51368 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.668235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:20.669896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:20.688972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.709414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.709414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.709414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.709414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.794027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:20.794457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:20.812237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 220/432] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:54:20.814085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.912232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 213/433] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:54:20.914660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:20.919417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.939953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37855c38-aa18-4281-836b-ecaea6ab9a94 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:20.941844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:20.991592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:20.992858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:20.993289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:20.993965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:20.995903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:20.998090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.001681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 221/434] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:21.009402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.035215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961790c1-cb3c-4977-bf87-78da9901a661 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.074346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.075291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.075635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.075998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] PATH_INFO: `/v3/projects/15f6d807458c48ecb1d58725592095f0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.077847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] RBAC: Authorizing `identity:get_project(project_id=15f6d807458c48ecb1d58725592095f0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.079346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.085539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 214/435] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:20 2022] GET /identity/v3/projects/15f6d807458c48ecb1d58725592095f0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:21.086797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.140143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f26bf-6836-462c-85af-7355fafc6796 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.182153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:21.191131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.191994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.192949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.193232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.194550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.195981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.202358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 222/436] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:21.204530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.206891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597c81b-7589-4f7f-912a-4231b6fac6ca None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.241521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:21.244011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.245014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.245401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.245722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.247039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.249973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.252351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 215/437] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:21.262248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.329824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f922c-0e8e-4447-b7fa-5f2f597ad144 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.368147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.368791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.369072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.369345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.370305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=b1643f7fcdff4280b6a98fb292c0e2cb, scope.project.id=379a41eb23be41efa1ea7937bcde4175) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:21.370599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.372152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a470f8-91ea-4891-821d-f190d6788f9a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.378493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 223/438] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=b1643f7fcdff4280b6a98fb292c0e2cb => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:21.379716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd265-7c2d-474f-986e-9a00c8852957 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.380007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd265-7c2d-474f-986e-9a00c8852957 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.380320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd265-7c2d-474f-986e-9a00c8852957 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.385639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 224/439] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:21.387255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.387538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.387813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.400009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363254e-9e59-46cd-a193-e0aff1215712 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.408214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:21.409450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:21.453880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.454964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.455460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.455944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.457031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=15f6d807458c48ecb1d58725592095f0) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:21.457578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.460014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.470589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 216/440] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=15f6d807458c48ecb1d58725592095f0&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:21.499426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 225/441] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:21.524784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.552510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:21.644400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d16288-cacf-4676-848d-00d7ef0c9bec None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.690069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.690881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.691171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.691446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.702838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:21.711205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:21.712099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:21.713639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 217/442] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:21.718161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.737053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:21.834884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11159bcf-8a70-4fae-8e67-81a07d37066f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:21.871937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:21.872662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.872947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.873831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.874589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:21.874902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:21.876346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88c93-a233-4bb4-95c3-2238066791b0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:21.884962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 226/443] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:21.890624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:21.907737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68d39e-92bd-49f8-83de-79e6bd3d7f12 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.908032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68d39e-92bd-49f8-83de-79e6bd3d7f12 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.908346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68d39e-92bd-49f8-83de-79e6bd3d7f12 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.909866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 227/444] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:21.914532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:21.914822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:21.915105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:21.935673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:21.937881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:21.991912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.015463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe0f6-9ded-4e41-8515-42bcb701e45e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.023212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 228/445] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:22.046474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.053882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.055011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.055517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.055973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.056543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.124835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.126309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.148669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf50448-c01f-4b72-bc3c-131082b31b70 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.181490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.182273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.182558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.182832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.193600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:22.201287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:22.202175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:22.203139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 229/446] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:22.203952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.208134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.223885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 218/447] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:22.229236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.274448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.307476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a085a2d-b953-4885-b1d8-c1b026ff9d06 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.325239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c24de05-9e3a-4408-b0b0-5685f74fa597 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.338771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.339424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.339703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.339979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.340756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:22.341056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.342934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.350962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 230/448] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:22.356448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.358740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.359595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.359895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.360193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] PATH_INFO: `/v3/projects/379a41eb23be41efa1ea7937bcde4175` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.361389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] RBAC: Authorizing `identity:get_project(project_id=379a41eb23be41efa1ea7937bcde4175)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.362914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.364533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 219/449] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/projects/379a41eb23be41efa1ea7937bcde4175 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:22.369825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.418077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.453674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c09324-06cd-41c7-bdb1-bcc890874ff2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.469534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405e6d-78ea-42f8-94db-ef90b937bd7c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.480966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.487712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.488379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.488667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.488948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.502708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.503551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.503833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.504122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] PATH_INFO: `/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.505303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] RBAC: Authorizing `identity:get_group(group_id=b0a4d929c01d4ddfa35c61077c0f5517)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.506743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.508390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 220/450] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/groups/b0a4d929c01d4ddfa35c61077c0f5517 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:22.514095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.553938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.555392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.613071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b5f8b-5990-4863-bf98-439a2a464e17 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.628973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.647304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.647945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.648251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.648532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.649358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] RBAC: Adding query filter params (group.id=b0a4d929c01d4ddfa35c61077c0f5517, role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=379a41eb23be41efa1ea7937bcde4175) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:22.649672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.651112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.652075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 231/451] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:54:22.657752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/role_assignments?group.id=b0a4d929c01d4ddfa35c61077c0f5517&scope.project.id=379a41eb23be41efa1ea7937bcde4175&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:22.658847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.689493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:22.765999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e832-1a91-450b-984d-e695b92c484b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.801372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.802226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.802503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.802776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] PATH_INFO: `/v3/projects/649b350e7f364263b37fbd34d0f07b2b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.804020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] RBAC: Authorizing `identity:get_project(project_id=649b350e7f364263b37fbd34d0f07b2b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.805440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.807115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/projects/649b350e7f364263b37fbd34d0f07b2b => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:22.812742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:22.914020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95034c-2ddb-43ef-a90b-51652076a8de None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:22.947237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:22.948092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:22.948415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:22.948700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] PATH_INFO: `/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:22.949888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] RBAC: Authorizing `identity:get_group(group_id=1d29ccfbde554fa7922c186a2fc0a24f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:22.951878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202024fb-183c-4b7c-81a8-4d9a9eca4616 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:22.953835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/groups/1d29ccfbde554fa7922c186a2fc0a24f => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:22.959448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:23.054143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:23.058202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdab00-9f0b-4a54-8fab-33667b7c8dcd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:23.091174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:23.091847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:23.092150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:23.092432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:23.093257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] RBAC: Adding query filter params (group.id=1d29ccfbde554fa7922c186a2fc0a24f, role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=649b350e7f364263b37fbd34d0f07b2b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:23.093569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:23.094994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211fa34-839d-4e59-bc13-0286d64472ed None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:23.100692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Mon Jun 20 06:54:22 2022] GET /identity/v3/role_assignments?group.id=1d29ccfbde554fa7922c186a2fc0a24f&scope.project.id=649b350e7f364263b37fbd34d0f07b2b&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:25.465649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b0e1-8085-44b2-a8c1-79998ac2a5fb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.466161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b0e1-8085-44b2-a8c1-79998ac2a5fb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.466264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9b0e1-8085-44b2-a8c1-79998ac2a5fb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.468884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:25.478269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c1adf-0739-48d8-8579-50c40965b427 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.478757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c1adf-0739-48d8-8579-50c40965b427 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.478932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c1adf-0739-48d8-8579-50c40965b427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.505139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-755c1adf-0739-48d8-8579-50c40965b427 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:25.506342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-755c1adf-0739-48d8-8579-50c40965b427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:25.601308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:25.643965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9b0e1-8085-44b2-a8c1-79998ac2a5fb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:25.723480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:25.793776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9b0e1-8085-44b2-a8c1-79998ac2a5fb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:25.796567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53bc8-4ba7-49ce-849a-71e6a30d715b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.797029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53bc8-4ba7-49ce-849a-71e6a30d715b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.797450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f53bc8-4ba7-49ce-849a-71e6a30d715b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.802587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 235/458] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:25.804074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f73808-58d7-4fc9-ac84-e887b32d28d1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.804608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f73808-58d7-4fc9-ac84-e887b32d28d1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.805144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f73808-58d7-4fc9-ac84-e887b32d28d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.833718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-42f73808-58d7-4fc9-ac84-e887b32d28d1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:25.836003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-42f73808-58d7-4fc9-ac84-e887b32d28d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:25.836870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:25.837895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.838370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.838844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.859503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:25.862555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:25.893822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:25.894719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:25.895588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:25.895588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:25.895588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:25.895588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:25.901842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 224/459] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:25.906187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:25.959656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 236/460] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:25.961354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7d0fb-c625-4172-8678-daa293260103 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.961845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7d0fb-c625-4172-8678-daa293260103 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.962315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7d0fb-c625-4172-8678-daa293260103 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:25.964644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 237/461] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:25.971909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:25.972229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:25.972509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.000406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:26.001894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:26.024874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.044272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.048844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b11b7-a9a4-48e9-9a12-c7f523160815 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.083787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 238/462] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:26.085059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.114210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.114886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.115172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.115450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.116208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:26.116521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.118267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.134639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 225/463] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:26.140581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.220510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c79e5-c320-46b1-9d46-ab59a3f503d6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.261805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.262613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.262900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.263180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.271478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b94dd3-35ae-4727-8c22-4f4e1522bfcc None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.275215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.276831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:26.298263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:26.299345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 239/464] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:26.300443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.305292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.323458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.324303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.324592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.324873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.335736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.337236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6a5d65a0-6e5c-4f55-b779-c718a788488b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:26.355476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:26.356372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:26.357790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:26.357790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:26.357790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:26.357790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:26.357790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 226/465] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:26.358468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bdd2c-c1c2-4844-bcfd-3ab601ad37b3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.358694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bdd2c-c1c2-4844-bcfd-3ab601ad37b3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.358974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bdd2c-c1c2-4844-bcfd-3ab601ad37b3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.364083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 227/466] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:26.364083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bdd2c-c1c2-4844-bcfd-3ab601ad37b3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.376812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.413020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b4061-7d63-4c61-8aa1-ff1e32a42b9c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.448413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.449087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.449374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.449653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.450358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.451831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.472427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 240/467] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:26 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:26.473563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.487553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bdd2c-c1c2-4844-bcfd-3ab601ad37b3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.527952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.528671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.529188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.529586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.530437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:26.530820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.532343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695accb-d739-4d8a-8f71-7a1af1dd773e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.546693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 228/468] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:26.552465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.552759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.553046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.554404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.572734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:26.573971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:26.621492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a3e58-f284-4722-b0c2-0c72ab7e0189 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.659381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 229/469] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:26.660858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.668355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.676136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.676136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.676136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.676136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.676495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:26.676495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.676495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.691051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 241/470] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:26.700161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.802553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b38144-f13f-44c7-bf36-09567d7393af None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.835218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078755c-81d2-405b-8631-9a70c28bd612 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:26.845748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.846426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.846715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.847353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.848075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.850197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.859101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:26.876465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 230/471] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:26 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:26.877318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:26.886012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:26.886830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:26.887121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:26.887450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:26.888214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:26.889836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ade7ea-e3fa-468e-ac28-8e888792646d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:26.933973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 242/472] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:26 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:54:26.949858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:27.002311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17a323-ded4-482e-92df-856d2103fc7f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.030957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb161-2cd9-4edb-91e7-c9933f1b6f8e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.031583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb161-2cd9-4edb-91e7-c9933f1b6f8e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.032096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddb161-2cd9-4edb-91e7-c9933f1b6f8e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.033941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 243/473] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:27.041426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.042262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.042559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.042845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.045424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.046043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.046552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.056604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:27.058093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:27.079144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:27.080072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:27.080980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:27.080980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:27.080980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:27.080980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:27.080980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 231/474] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:27.082959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:27.084506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:27.086745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:27.200198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 244/475] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:27.224838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:27.234787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88b7dc-a0a3-43a7-acd6-38d0df637f22 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.245821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:27.278108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.278781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.279616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.279902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.280675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:27.280982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:27.282465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:27.295238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 232/476] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:27.300306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:27.342846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c2ca3-3184-43dd-8b3a-030fcea04324 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.350935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:27.380953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.382022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.382519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.383003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.383955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:27.385587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:27.394681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.sql.core [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 06:54:27.394681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 06:54:27.394681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [parameters: {'id': '5d2d86cefbea4dbf93790027dbc5cf51', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 06:54:27.394681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80136) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 06:54:27.406894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9a0526-5559-4435-b34b-eac144c4e888 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jun 20 06:54:27.415092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return ref.to_dict() Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:54:27.416490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application yield resource Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise self.value Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application session.commit() Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 06:54:27.417469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.flush() Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application compat.raise_( Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 06:54:27.418396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:54:27.419307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:54:27.420205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [parameters: {'id': '5d2d86cefbea4dbf93790027dbc5cf51', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:27.421022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 06:54:27.421829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.426041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 245/477] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Jun 20 06:54:27 2022] POST /identity/v3/roles => generated 144 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:27.431911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:27.470865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.472930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.473441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.474302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.476844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:27.478531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:27.521564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 233/478] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:27 2022] POST /identity/v3/users => generated 249 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:54:27.593228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02690853-08dc-457d-a3ec-9885564a8dac None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.632752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.633990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.634476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.634950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.647960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bba44866-b91e-42e4-ba2a-6e89636db79d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:27.656698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:27.657582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 06:54:27.658413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:27.661298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 246/479] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:54:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:27.669266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:27.717164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:27.814450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d68e0-cfb9-4cc9-9a35-1f3bea533421 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:27.849943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:27.850538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:27.850824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:27.851212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:27.851967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:27.852298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:27.853667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b158c-4594-4d95-8dac-8bedfd2dd69b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:27.862904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 234/480] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:29.165930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f53d-02a6-4970-b4ca-ef06a0275b28 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.166443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f53d-02a6-4970-b4ca-ef06a0275b28 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.166519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f53d-02a6-4970-b4ca-ef06a0275b28 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.168131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 247/481] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:29.181496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0b0e-68b8-46b9-8e57-32f649c1a3f2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.182390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0b0e-68b8-46b9-8e57-32f649c1a3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.182864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a0b0e-68b8-46b9-8e57-32f649c1a3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.205099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-105a0b0e-68b8-46b9-8e57-32f649c1a3f2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:29.206586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-105a0b0e-68b8-46b9-8e57-32f649c1a3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:29.285691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 235/482] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:29.321076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f53d-02a6-4970-b4ca-ef06a0275b28 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:29.365634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:29.396533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35194a9e-16b3-4838-a0a2-8003c62a9c58 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.396851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35194a9e-16b3-4838-a0a2-8003c62a9c58 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.397104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35194a9e-16b3-4838-a0a2-8003c62a9c58 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.398707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 236/483] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:29.404704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e45ff-c248-4acb-aab7-5d4227a9cc89 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.404990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e45ff-c248-4acb-aab7-5d4227a9cc89 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.405271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e45ff-c248-4acb-aab7-5d4227a9cc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.450187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f53d-02a6-4970-b4ca-ef06a0275b28 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:29.453314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0e6e45ff-c248-4acb-aab7-5d4227a9cc89 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:29.454529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0e6e45ff-c248-4acb-aab7-5d4227a9cc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:29.506054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:29.509458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:29.510283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.511227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.511227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.529135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:29.543069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:29.543971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:29.545825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 248/484] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:29.550180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:29.651719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:29.667408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 237/485] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:29.668680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e6e-3602-4ad0-b582-e3556b2f8e3e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.669182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e6e-3602-4ad0-b582-e3556b2f8e3e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.670195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c8e6e-3602-4ad0-b582-e3556b2f8e3e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.675755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 238/486] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:29.677168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.677600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1c806-5b7e-43ce-a7af-89b598690050 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:29.677920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.678400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.713248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:29.715127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:29.767096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:29.767796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:29.768056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:29.768361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:29.769126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:29.769438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:29.770864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:29.786403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 249/487] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:29.787337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:29.804628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 239/488] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:29.809558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:29.960556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de34ab4-e7e2-4bb3-be53-a32329971b83 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:29.973357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.001801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.003389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.003881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.004397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.017592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:30.026828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.027724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:30.028576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:30.032008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 250/489] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.033094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.047126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f4940-6a4f-440b-8c30-3918cb429ba8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.051281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.114961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.116172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.116440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.116718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.140712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.146477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.159916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42950611-ad93-4152-b667-bf2ad15fcbc8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:30.174479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.175400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.176298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:30.176298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:30.176298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.176298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:30.180605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 240/490] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.181437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.207741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.209859 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.210356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.211125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.225064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:30.237296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.238197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:30.238996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:30.246524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 251/491] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.246524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.294937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b465aed-0b2d-46dd-a4d7-5c487cd7c263 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.332581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.337242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.340823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.341557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.341841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.342118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.342876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:30.343186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.345015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.358386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 241/492] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:30.358386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.381650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a26976-ea96-46e9-aede-4ced4f314aad None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.418473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.419141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.419963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.420286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.421127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:30.421441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.422939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf55c-b366-47ad-9464-4455b26fe25d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 252/493] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:30.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199c33-0064-480e-9de4-3c8d84af9890 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199c33-0064-480e-9de4-3c8d84af9890 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32199c33-0064-480e-9de4-3c8d84af9890 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 253/494] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:30.441542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199c33-0064-480e-9de4-3c8d84af9890 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.465820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d649d-b1f2-4d4b-aeef-d6d5ea23d285 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.500268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.500931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.501218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.501493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.502266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:30.502579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.503952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.511728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 242/495] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:30.512541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.542554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32199c33-0064-480e-9de4-3c8d84af9890 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.568252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.572158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.577437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.578219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.578506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.578776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.588620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.590065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-404373fb-8f7f-48c1-9779-fcf30d7ab7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:30.608348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.609275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 254/496] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.611821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.622426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeab4e7-635d-40f6-84ae-40f5df3754ba None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.629345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:30.630525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:30.659226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.660007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.660378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.660598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.667882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.669350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6ac4ff5d-d236-4314-9266-3bcb700e1039 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:30.683948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:30.686868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 243/497] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.687581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649ea90-a8f8-4b29-ae73-89912e0249af None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.687581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649ea90-a8f8-4b29-ae73-89912e0249af None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.687734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649ea90-a8f8-4b29-ae73-89912e0249af None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.688970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 244/498] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:30.689644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649ea90-a8f8-4b29-ae73-89912e0249af None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.715236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 255/499] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:30.715236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.796593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649ea90-a8f8-4b29-ae73-89912e0249af None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.809708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489ff3-ac64-4527-9bd0-805969ef3bf3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:30.809849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.824793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:30.831302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.832084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.832402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.832681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.842991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.844520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.845120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:30.845776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.846062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.846335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.847054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:30.847366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:30.848781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:30.860216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 256/500] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:30.860999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-568731ba-2b93-4528-a0a6-fd1682810a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:30.861750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:30.862583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:30.863489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:30.863489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:30.863489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:30.863489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:30.863489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 245/501] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:30.864083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:30.864396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:30.864654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:30.882055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:30.883254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:30.949787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 246/502] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:30.951121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:30.970703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba958c5-3423-4a8b-8606-a396318b25aa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.004933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.005597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.005880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.006160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.006996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:31.007314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.008787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.044032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.046389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 257/503] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:31.047264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.052456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de7dcff-2241-467f-8f5c-c55d75e6d38f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.079333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.086291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.087075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.087356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.087627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.098111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:31.105706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.106587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:31.107470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 247/504] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.108158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.150397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f5a02-4600-4ea7-8ab0-800a6090e1ba None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.183601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.184421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.184711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.185023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.191901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.193384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.207451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a576a-0699-4982-aedd-74f3faa26132 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:31.209612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.211002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 258/505] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.212089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.243797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.244480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.244768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.245048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.245774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:31.246081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.247991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.259444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 248/506] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:31.260969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.278339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.301877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.311629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afc94f-790c-4d30-b462-3db9aa3f0bd0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.344554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.345333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.345613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.345886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.356150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.357646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.358012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac2018-68e8-4733-b13c-197f3ff478ec None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:31.375139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.376188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 259/507] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.377279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.393110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.393756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.394046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.394327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.465696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.467145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.476353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdcde2e-ad4d-4d0b-a795-232ecc291722 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.483439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.508708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.509368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.509653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.509930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.510678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:31.510994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.512396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.520847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 260/508] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:31.521619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.531564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.566124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 249/509] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:31.566870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.621320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b7390-3aaa-48b8-ae0f-a3cbe24b23e3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.654277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.654957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.655240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.655518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.656373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:31.656684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.658125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.664019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddf252-f402-4272-9a62-2a7033f7510c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.679254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 261/510] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:31.679254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.680927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.699726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.700552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.700841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.701119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.708449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.709956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:31.727512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.728582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:31.729511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 250/511] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.730099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.741307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.779512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f5a09-6bc4-4ffa-b7ce-9377987b068d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.813274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.813926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.814206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.814487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.815203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:31.815512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.816964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.828233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 262/512] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:31.828970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14077582-61db-4bbe-8465-b28046c91fc7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.829069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.865511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.866295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.866599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.866847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.877518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.878971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:31.896305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.897258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.898131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:31.898131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:31.898131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.898131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:31.898131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 251/513] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.898395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:31.928964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b053f-b706-4ee0-a1f8-4944bdeaf99a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:31.931562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.952716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:31.967084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:31.967865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:31.968184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:31.968464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:31.978794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:31.980288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:31.996734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:31.998113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 263/514] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:31.999182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.007175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c416e-5dfe-46cd-b1ed-4be2ac935df2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.040902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.041537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.041821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.042096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.099567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14dd1a-1758-4619-823e-33d2f7018478 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.107202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.108673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.132580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.133240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.133529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.133807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.134328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.134647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:32.134965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.136428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.154050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 264/515] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:32.154822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.190468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.207026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 252/516] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:32.207859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.256306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdb22d-0081-441f-9337-c5927f09637d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.290043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.290985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.291272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.291554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.292301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.294144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.304661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382bc27-4aa3-4ae1-b20c-d4ae00a85231 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.313200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 265/517] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:32 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:32.313992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.334695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.346336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.347008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.347297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.347577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.348351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:32.348662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.350158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.363449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 253/518] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:32.364542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.438518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.452055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938649-3df7-45b7-89fc-61298bc41651 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.483712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c30f5b-a9fd-479d-8760-f63203cdbd16 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.498552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.499306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.499688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.499972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.500822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:32.501050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.502451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.519896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 266/519] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:32.520793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.523393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.524045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.524380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.524658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.589629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.605986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.608046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.626854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba66e3-f841-41ff-80d4-532fde916445 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.661404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.662332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.662707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.663097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.674401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.675966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:32.694673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:32.695770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:32.696875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:32.696875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:32.696875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:32.696875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:32.698079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 267/520] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:32.704202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.706276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.722334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 254/521] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:32.722334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.819444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:32.826872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999689c-1ae0-49be-8d05-e77824f6a019 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.832669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c48d-d5b1-4363-a346-cba37dda23b5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:32.864795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.865509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.865794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.866072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.868260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:32.869561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:32.869879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:32.870157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:32.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.875303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.877320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:32.879246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:32.889429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:32.890333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:32.891514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 268/522] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:32.892079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:32.898277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:32.899114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 255/523] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:32.905451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:32.962574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.006548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411e7e0-79f8-4364-9ed7-9548a5a2b762 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.009711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95521ded-5095-4351-9ebd-c93b6a61c0ca None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.044982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.045206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.045767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.045867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.046053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.046148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.046330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.046423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.047051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.047357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.048757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.057815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.059256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.061490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 269/524] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.062559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.071686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:33.082220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:33.083185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:33.084231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:33.084231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:33.084231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:33.084231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:33.085853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 256/525] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:33.090887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.171519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1da3cc-3abf-4f1b-bcb8-853b02f0339b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.190275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a5f04-3898-4145-872f-87f9c0607acd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.194414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.206202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.206855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.207135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.207409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.208258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.208565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.209986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.225739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.226378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.226665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.226946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.227689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 270/526] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.227795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.228026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.229499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.233162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.253076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 257/527] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.253076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.313518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.339948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc7480-1fa4-4cb7-96df-922dd644083f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.355668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0345c61-3407-4054-a7b4-59012990e378 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.373764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.374615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.374962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.375300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.381885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.383343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.392166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.392899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.393250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.393592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.394380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.394763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.396211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:33.397545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:33.398612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 271/528] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:33.404212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.415641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 258/529] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.416345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.456096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.528333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d76cb-e0af-43e1-ab23-22cabcd4a871 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.529393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8a942-09c2-4c72-b243-97c1c9540216 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.550418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.565567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.565741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.566221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.566507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.566600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.566785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.567046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.567417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.567610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.567924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.569385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.575944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 259/530] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.576533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.683887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.686236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.707494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739b69e-aaa8-49f9-9d88-e6ef69435032 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.748013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.757067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.758245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.759019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.759398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.760320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.760710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.762973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.777747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 260/531] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.779307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.824147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 272/532] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:33.825997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.860504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:33.897821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb69ef5-62a9-4344-9bca-77fe0909aea7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.933271 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75eb72e-1108-405d-8b07-4a5a69821114 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:33.935935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.937025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.937538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.938027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.946427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.948502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:33.964819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:33.965703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:33.967205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:33.969519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 261/533] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:33.970769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.971754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:33.981034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:33.981337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:33.981607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:33.982494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=468b120f57fe4807b969c390cbcabfbb) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:33.982801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:33.984960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:33.993013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 273/534] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/role_assignments?user.id=468b120f57fe4807b969c390cbcabfbb&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:33.993873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:33.998370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.130286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6fc05-411e-4053-a8fd-48b319232d82 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.134854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.174762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e70d2-9ef7-4a45-abec-2394a7f1517f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.200875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.202179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.202780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.203117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.215141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.215834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.217262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.217368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.217811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.218263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.219300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:34.219797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.221430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:34.237458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:34.238355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:34.239202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:34.239202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:34.239202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:34.239202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:34.239202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 262/535] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:34.244225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 274/536] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:34.249354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.250339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.358595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.359924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc38b8-bec1-42ab-b23c-f412c0ed6d24 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.398652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9fbdd-f11f-422c-aeb0-5f7c119869f5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.409756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.410438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.410719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.410994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.411723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:34.412025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.413428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.434179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 263/537] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:34.437731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.438490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.438927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.439541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.440349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.440828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e318b1b6386c4a2d9e7f324e8fc2cfe6) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:34.441267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.442987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8e329-368f-407e-bfda-a49c330bca3c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.450999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 275/538] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/role_assignments?user.id=e318b1b6386c4a2d9e7f324e8fc2cfe6&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:34.477500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.484024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a00dc-bb76-4cde-86b7-15413997b4f0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.484373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a00dc-bb76-4cde-86b7-15413997b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.484635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a00dc-bb76-4cde-86b7-15413997b4f0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.486288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 276/539] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:34.492419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.493009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.493585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.512517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:34.514661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:34.613366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 277/540] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:34.642686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.647730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e03fe4-49f5-4824-be63-ea9cb93b40b1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.705091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.707882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.708214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.708493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.717408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.718875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:34.735213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:34.736080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:34.736970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 264/541] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:34.743246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.773812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.822982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:34.862704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a8146-9e5e-458e-a412-fd01333304b5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.866835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274e9e1-6c37-4cbe-b008-cf2724756fe0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:34.932851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.933548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.933834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.934116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.935469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:34.935785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:34.937265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:34.943659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:34.944574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:34.944860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:34.945136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:34.963507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:34.971785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:34.972680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:34.973472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:34.974608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 265/542] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:34.981370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:34.985049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 278/543] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:34.996738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:35.152667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465a70b-ba5b-48dc-a076-e7c6ef8808bc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:35.170141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:35.188600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:35.189283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.189568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.189843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.190670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=014dfb1baeb24fa7ab38923a6962db4b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:35.190981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:35.192437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:35.194212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f1d10-b4a4-427c-9091-4e1187f788ff None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:35.198458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 279/544] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/role_assignments?user.id=014dfb1baeb24fa7ab38923a6962db4b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:35.276247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:35.276925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.277208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.277484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.278234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:35.278573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:35.279953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:35.285334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:35.294763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 266/545] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:35.300098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:35.466410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab464e7a-21b3-4942-afc3-b117e51ff132 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:35.508823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:35.510015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.510502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.510973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.526001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:35.527858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:35.545874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:35.546745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:35.547602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:35.547602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:35.547602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:35.547602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:35.549840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 280/546] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:35.557191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:35.720261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902af629-f202-4a41-ade4-20e84b396600 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:35.774662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:35.775331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.775615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.775893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.776649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:35.776957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:35.778903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59227e6-e1b7-411f-a266-ba33b86b135b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:35.801484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 267/547] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:35.806529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:35.894564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:35.952617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5754-5ecd-4db5-9c9e-73615c5f758b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.952945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5754-5ecd-4db5-9c9e-73615c5f758b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.953175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db5754-5ecd-4db5-9c9e-73615c5f758b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.954587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 268/548] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:35.960074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:35.963029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:35.963029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:35.990537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ee09e-6cac-4adc-ab3a-c7362daeb4a9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:35.993727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:35.994485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:36.025852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.026939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.027398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.027850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.036628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.038279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:36.053175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:36.054602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:36.055677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:36.058337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 281/549] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:36.066968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.108512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.125993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 269/550] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:36.161466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.171803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.178510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dae25e-1dbf-4d65-8dc1-73fddfab55d8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.220821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.221494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.221786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.222065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.222902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:36.223212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.224674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.258782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 282/551] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:36.263813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.294716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f70027d-eedf-4fb4-b488-05c3d6fda5b8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.344548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.345360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.345777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.346174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.373185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:36.389516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:36.390519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:36.392298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 270/552] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:36.400378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.447720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd54b-2ef4-4751-bee6-8d1287e45260 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.491481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.510051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.511512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.512580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.513057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.545454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.556022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c1bff-14fb-4785-a7f4-1ea5ce408840 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.595647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.597105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.597605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.598059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.598982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:36.599468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.601057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77d5b-1876-48d9-bc44-61b8eca948a2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.603806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.605297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.610974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 271/553] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:36.612601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c0e8-955d-4175-a72d-86d8c84381f3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.613109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c0e8-955d-4175-a72d-86d8c84381f3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.613607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422c0e8-955d-4175-a72d-86d8c84381f3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.619335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 272/554] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:36.620702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.621203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.621683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.642280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:36.644756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:36.727300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 283/555] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 06:54:36.727300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.749542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 273/556] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:36.749542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.804509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.809578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:36.867204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee65472-8511-44b9-93d5-b2a04965b2f9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.870396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822a7bd-eecd-4f77-ab65-a9a764dcfa7c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:36.904656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.905583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.905984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.906351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.913700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:36.914679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:36.915502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:36.915932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:36.917253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.918897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.928299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:36.930355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:36.941380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:36.942413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:36.943299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:36.943299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:36.943299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:36.943299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:36.948144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 274/557] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:36.948144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:36.956800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:36.958130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:36.959005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:36.959005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:36.959005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:36.959005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:36.959005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 284/558] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:36.959287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.065935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.067789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04addb25-7aab-4768-bdb7-e12e52d89bb8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.070326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.076736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710aca-6a74-4e74-a09c-6a7e7b23cc69 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.107389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.108045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.108357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.108656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.109350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.109656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.111016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.120491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.121276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.121553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.121839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.122393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 285/559] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:37.123086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.136682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-98eb89f4-e66d-4334-b789-6b33f9459410 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:37.145106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:37.146157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:37.147039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:37.151249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 275/560] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:37.152580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4a5f-5376-4ec1-a9bb-8394c01c48cc None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.152871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4a5f-5376-4ec1-a9bb-8394c01c48cc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.153160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c4a5f-5376-4ec1-a9bb-8394c01c48cc None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.155237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 276/561] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:37.155937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4a5f-5376-4ec1-a9bb-8394c01c48cc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.227422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f587-1dda-42d3-800e-39928053b9f6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.261683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c4a5f-5376-4ec1-a9bb-8394c01c48cc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.266920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.267832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.268335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.268797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.270120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.270615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.272569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.286009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 286/562] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:37.286835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.300809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.302001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.302540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.302919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.311142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.312473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.313980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-61149184-7c17-414d-81c1-c2f300cfdc18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:37.331853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:37.332831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:37.333742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 277/563] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:37.334264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536481-3fd1-4184-835d-19df15acb04a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.335267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536481-3fd1-4184-835d-19df15acb04a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.335267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91536481-3fd1-4184-835d-19df15acb04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.352256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-91536481-3fd1-4184-835d-19df15acb04a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:37.354109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-91536481-3fd1-4184-835d-19df15acb04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:37.364539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.398613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447968e-dcf5-4961-b612-14054f13d095 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.432664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 278/564] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:37.433492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91536481-3fd1-4184-835d-19df15acb04a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.441054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.441969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.442317 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.442667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.443640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.444037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.445688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.457410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 287/565] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:37.458654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.543854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.548632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91536481-3fd1-4184-835d-19df15acb04a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.560338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324a99-ac9c-492c-8341-f03490349562 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.586005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.586719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.587001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.587445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.594488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.594865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.595519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.595803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.595908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.596344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.596891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.597201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.598627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:37.611038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:37.612012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 279/566] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:37.613330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.616671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 288/567] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:37.616973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.617950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.726962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9168fb-d9fc-46f2-bbaa-a32a12714237 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.733231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af7af2-d799-4abd-bd7b-726705b8020b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.768951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.769865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.770153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.770541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.777654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.778468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.778755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.779031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.781236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:37.788933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:37.790100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:37.791136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 289/568] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:37.791845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.799448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.800877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.800999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:37.820780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:37.821696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:37.822604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:37.822604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:37.822604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:37.822604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:37.822604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 280/569] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:37.822880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.880513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:37.896072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9eeb43-0063-48d9-962d-a379a86fa152 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.924756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037019-fd15-47dc-afb7-b3c36fa7f3b2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:37.928925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.929561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.929841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.930115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.930930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=601afd44d1f54f939e87511ee463cc4a) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.931241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.932699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf9a9e-707b-43ed-9eeb-16ca775545b8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:37.938757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 290/570] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/role_assignments?user.id=601afd44d1f54f939e87511ee463cc4a&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:37.939922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967281-9e62-4ea7-8fc9-de8d84b5163f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.940232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967281-9e62-4ea7-8fc9-de8d84b5163f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.940510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3967281-9e62-4ea7-8fc9-de8d84b5163f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.941742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 291/571] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:37.942446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967281-9e62-4ea7-8fc9-de8d84b5163f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:37.969397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:37.970089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:37.970422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:37.970648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:37.971513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:37.971834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:37.973378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.015879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 281/572] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.016864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.027485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.061229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3967281-9e62-4ea7-8fc9-de8d84b5163f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.101415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.102089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.102375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.103289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.103434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.103712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.105135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c3520-e7df-4def-bda8-c1103eea9ef7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.114938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 292/573] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.119649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.119649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.119649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.136471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:38.136614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c68fe1-b5e3-4481-ac06-a6b007fac57d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.138432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:38.151099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.174135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.174895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.175252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.175606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.176451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.176829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.178310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.195450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 282/574] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.195450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.220266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 293/575] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:38.220266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.294234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.302693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419121a-1ba3-4485-b0a5-ec214c42b2f8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.335615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f12b64-bf16-42e1-90da-45999e077911 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.343562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.344660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.345466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.345932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.358075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.359777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.370565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.371362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.371651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.371955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:38.378899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:38.379861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:38.381011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:38.381011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:38.381011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:38.381011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:38.382616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 283/576] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:38.383406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.383709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.385160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:38.407548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:38.408712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 294/577] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:38.409622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.418663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.502496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a0ff0-1022-4d23-82fa-bd564f67e4a5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.516044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5c1a8-84dc-49c1-be2b-cbf6c3d9674e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.531986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.537794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.538599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.538889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.539163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.549032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.550489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.554725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.555514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.555794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.556068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:38.565468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:38.566406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:38.567397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 284/578] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:38.568719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.571925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:38.579689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:38.580649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 295/579] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:38.582612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.659186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.676843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b33089-ba8a-40b5-83ec-2d2f782f02e6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.711717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c14447-110a-458d-860a-443f4fdfe139 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.726277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.726953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.727228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.727510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.728274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.728591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.729968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.742973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 285/580] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.743841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.756349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.757009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.757294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.757570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.758297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.758604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.759987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.772847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 296/581] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.773630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.815524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.854084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7856d7e-c2c1-41bb-a804-67113a4925a8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.886255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f870c-df0f-4e7b-a497-d7fa6b778314 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:38.892175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.893084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.893606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.894481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.895514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.896387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.898613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.923290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 286/582] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.923290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:38.925549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:38.926185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:38.926294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:38.926475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:38.926743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:38.927490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:38.927794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:38.929206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:38.937547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 297/583] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:38.938327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.039744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee01a4-9fa4-42b6-b742-77ec927f3296 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.050190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.059097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950bd25f-b6e7-4ceb-b3d1-5574709c0a5e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.077768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.078562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.078841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.079109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.086074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.087520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.093820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.094615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.094905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.095184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:39.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:39.102445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:39.103357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 287/584] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:39.103856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.108442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.109898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:39.124192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:39.125134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 298/585] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:39.131180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.172301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.212441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3c12-5da4-40ef-96ae-37473292c407 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.234006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452991b-a50b-474e-8e5d-a0fbf1caa8dc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.246133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.246941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.247762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.248040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.259013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.260615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.268912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.269686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.269971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.270253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:39.278204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:39.279084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:39.280030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:39.280030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:39.280030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.280030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:39.280030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 288/586] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:39.281250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.285543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.287021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:39.304824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:39.305735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:39.306783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:39.306783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:39.306783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.306783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:39.306783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 299/587] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:39.307562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.320038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.392209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabd495-24b4-482e-b903-0c388e324d9f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.406879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.423717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4356480-fb7f-43f3-93a7-0c21c769ff83 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.427906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add010-b017-4f51-84e2-338318718edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.428596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add010-b017-4f51-84e2-338318718edf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.428885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add010-b017-4f51-84e2-338318718edf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.429164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add010-b017-4f51-84e2-338318718edf None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.429952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04add010-b017-4f51-84e2-338318718edf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:39.430265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04add010-b017-4f51-84e2-338318718edf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.431708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04add010-b017-4f51-84e2-338318718edf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.450062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 289/588] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:39.450062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add010-b017-4f51-84e2-338318718edf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.466152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.466851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.467141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.467518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.468281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:39.468593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.470036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.492655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 300/589] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:39.492655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.557160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04add010-b017-4f51-84e2-338318718edf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.563618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.592867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.593527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.593807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.594080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.594805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:39.595110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.596498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.602865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0b015-ddc3-4a53-b749-c206294873a8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.610196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 290/590] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:39.611086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.646885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.647547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.647831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.648140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.648872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:39.649180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.650556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.651363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.668209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 301/591] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:39.669107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.730219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a30c2-fb37-4a35-8818-a536415f0988 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.772879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.773770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.774055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.774332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.784308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869dafeb-6988-4bdd-bde1-9db4a9adf972 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.790058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=e19cf623e22c485fb1bb8f0ada7114b9, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.791515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.805010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 291/592] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:39 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:39.806083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.827479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.859748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.860882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.861175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.861453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.878510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.880377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:39.903704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:39.906218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:39.907638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 302/593] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:39.909033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:39.930428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aba9c-6f58-43e7-8be5-c35cf12777e7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:39.977038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:39.977939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:39.978297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:39.978643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:39.987975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:39.988612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:40.005401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:40.006479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 292/594] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:40.008834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.057275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb078e7d-6482-469b-800a-04d419506bad None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.091661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.092487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.092758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.093031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.101088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.102580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-64c36bec-7587-4135-b37e-6d0ed2fcb044 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:40.117331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:40.118357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:40.119312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:40.119842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e468bf-d408-4260-bacc-01af25a82155 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.119916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 303/595] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:40.120630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a04b2-7b49-4880-bca9-156a59895408 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.120630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a04b2-7b49-4880-bca9-156a59895408 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.120804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a04b2-7b49-4880-bca9-156a59895408 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.126662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 304/596] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:40.126662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a04b2-7b49-4880-bca9-156a59895408 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.157237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.157905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.158186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.158462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.159181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:40.159485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.161484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.166425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.174860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 293/597] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:40.175695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.181485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.238352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a04b2-7b49-4880-bca9-156a59895408 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.276806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.277473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.277788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.278044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.279382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:40.279703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.281263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1624643-7416-4768-a22a-c0d3ac22255b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.285746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7159a0-29ef-403d-bedd-c5d47341f245 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.293676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 305/598] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:40.299922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.300246 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.300538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.319276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:40.320546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:40.322208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.322878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.323160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.323435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.324298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:40.324605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.326059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.343981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 294/599] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:40.344832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.396714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 306/600] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:40.397834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.415326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.435948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.456668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fd9e7-2835-4be5-b17d-b76ee2065546 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.494164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.495447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.495862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.496170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.503643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.505235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.512597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7abf5b-3031-4edb-98ca-e098335c0b83 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:40.519999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:40.520974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:40.521874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 295/601] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:40.522747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.553170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.554050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.554337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.554611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/468b120f57fe4807b969c390cbcabfbb/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.570299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=468b120f57fe4807b969c390cbcabfbb, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.572268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.586298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 307/602] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:40 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/468b120f57fe4807b969c390cbcabfbb/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:40.589297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.636057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97481bb2-3e09-450b-bbb8-ceb4b348017e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.665079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.678448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.679309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.679571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.679846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.692852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.694231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:40.713374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:40.714343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:40.715249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:40.715249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:40.715249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:40.715249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:40.715249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 296/603] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:40.716007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.723773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.751956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7819489-cb2a-4f24-8024-e9aa6d3c5899 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.788879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.790076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.790507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.790942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.804703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:40.813062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:40.814011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:40.814895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:40.819270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 308/604] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:40.820092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.841786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d78b6-8723-4d84-bf29-d6f0caa7dc12 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.891234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.892546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.893610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.894091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.895367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:40.896154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.898691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.915031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 297/605] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:40.916714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:40.929708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1d32-d810-429d-a2c8-5dd44e20dc64 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:40.961037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.969173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:40.971187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:40.971739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:40.972423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:40.973426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:40.974062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:40.975698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:40.986030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:40.994163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 309/606] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:40.995393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.045735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5380-bbf1-4ee8-b1f1-1bcfda82083c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.085197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.086231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.086839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.087354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.088454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:41.089077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.090799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.100340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 298/607] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:41.101567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.109885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c0611-5a22-47b3-9e59-430fedcadb63 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.153041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.154197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.154481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.154757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e318b1b6386c4a2d9e7f324e8fc2cfe6/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.172439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=e318b1b6386c4a2d9e7f324e8fc2cfe6, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.174497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.191823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 310/608] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:40 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e318b1b6386c4a2d9e7f324e8fc2cfe6/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:41.192867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.210353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:41.218177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1741853-20c5-4a44-9f6e-610e6d4e187f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.256065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:41.260721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.261820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.262394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.264418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.277505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.279018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:41.323116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:41.324164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:41.330075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:41.336777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 299/609] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:41.344060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.378085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980723d2-4c80-4901-9dde-23221e98fc9d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.429812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.431012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.431497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.431971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.446552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.448445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:41.468487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:41.469435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:41.470330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:41.470330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:41.470330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:41.470330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:41.472737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 311/610] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:41.480160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b9d1f-45ae-4ad4-9b78-552de1af4fe9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.481985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.525008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.525503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.525779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.526054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.526880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:41.527150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.528665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.551187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 300/611] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:41.557052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.592974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:41.654794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:41.659811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efa2b5-2d80-49b0-821c-dda010ed802b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.729888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.730940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.732165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.732647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.734708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:41.735220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.737579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.759276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 312/612] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:41.767138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:41.772687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42891cca-b781-42f3-af4b-90832fdebf68 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.850733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.851556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.851841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.852135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/014dfb1baeb24fa7ab38923a6962db4b/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:41.883563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=014dfb1baeb24fa7ab38923a6962db4b, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:41.885056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:41.911869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 301/613] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:41 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/014dfb1baeb24fa7ab38923a6962db4b/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:54:41.946523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9708d-b580-4546-94df-2284fea5ead5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:41.990303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:41.991108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:41.991451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:41.991729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.000218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:42.001780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b88c52df-f9a2-498d-9f19-816787555925 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:42.017254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:42.018264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:42.019204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 313/614] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:42.024477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.101987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd743-1429-4165-b4df-8036d459a451 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.102290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd743-1429-4165-b4df-8036d459a451 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.102464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdd743-1429-4165-b4df-8036d459a451 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.103956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 314/615] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:42.109646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.110261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.110743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.129142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:42.130900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:42.136796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.158373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.213078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f178380-4f8f-4590-9364-64f2affd905f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:42.235960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 315/616] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:42.258516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:42.259343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.259755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.260035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.260922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:42.261233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:42.262709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:42.265045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.286828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 302/617] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:42.291158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.438062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90741e50-a9b1-4478-9d4f-7af421963017 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:42.479720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12ff1b-e46d-4bcd-9862-13497b3eb870 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:42.511931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:42.512805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.513085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.513359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.530019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.539763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:42.552808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:42.553793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:42.554952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 316/618] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:42.561050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:42.561870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.562155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.562431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.563844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.581486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.586653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:42.588424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:42.614547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:42.615514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:42.616580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:42.616580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:42.616580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:42.616580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:42.623388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 303/619] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:42.624890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.745840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb561-20f2-4cb2-a52a-9c8f04faa07c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:42.800941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0621763-ea13-4572-847f-b54dc0896dbd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:42.810854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:42.811530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.811812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.812088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.812872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:42.813179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:42.814549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:42.828790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 317/620] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:42.834609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.855146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:42.855822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.856132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.856420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.857258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:42.857668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:42.859141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da683bb-81ff-468a-86c9-510a094da68f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:42.875120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 304/621] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:42.876714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714b3f2-7030-4e00-8e6d-55830a999eaf None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:42.876830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714b3f2-7030-4e00-8e6d-55830a999eaf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:42.877259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714b3f2-7030-4e00-8e6d-55830a999eaf None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:42.879287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 305/622] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:42.882256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714b3f2-7030-4e00-8e6d-55830a999eaf None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:42.924426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.925314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:42.961610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87848484-b79e-4985-97c4-c5e8a96e250f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.008080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.008904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.009121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714b3f2-7030-4e00-8e6d-55830a999eaf None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.009239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.009459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.024295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.026483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.046659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.047755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.048294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.048782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0e4c8acc-26c0-40ac-b51b-32e514661d41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:43.049410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:43.050299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:43.051198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:43.051198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:43.051198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:43.051198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:43.052175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 318/623] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:43.053577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.054072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.054575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.066208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.067880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.084137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:43.085357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:43.091576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:43.092599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:43.093550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:43.101160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 306/624] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:43.101160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.176314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 319/625] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:43.177176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.207680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.214805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab3a51-7b5d-437e-b967-b7200f66af51 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.253003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.253967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.254277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.254791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.255289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.256251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:43.256818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.258508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.276043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 307/626] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:43.277329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.305044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9f950-18e3-487e-807c-17fb6288c228 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.343009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.343690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.344014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.344329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.345204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:43.345514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.347073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.364164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 320/627] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:43.364164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.398024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a49b0a-f1ac-41ff-9f8c-aca004669354 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.442258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.443087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.443372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.443654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.458710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:43.468150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:43.469124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:43.470004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:43.472250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 308/628] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:43.473259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.497595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.498986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61103d-18f1-4067-b3a1-e7132cfec073 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.520090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.545083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.545881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.546164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.546437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.554091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.555526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:43.569800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:43.570786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 321/629] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:43.571804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.598507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb7fdb7-ca87-4aa1-a1db-ed71b1ad3e94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.633847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.634758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.635025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.635301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/601afd44d1f54f939e87511ee463cc4a/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.650818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=601afd44d1f54f939e87511ee463cc4a, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.652302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaed19f-56c5-4196-ac9c-f766121d928c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 309/630] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:43 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/601afd44d1f54f939e87511ee463cc4a/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dc786-be9a-4b9d-bf2b-a6bc93ce07ad None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dc786-be9a-4b9d-bf2b-a6bc93ce07ad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dc786-be9a-4b9d-bf2b-a6bc93ce07ad None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 310/631] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:43.669669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dc786-be9a-4b9d-bf2b-a6bc93ce07ad None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.690861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fa510-807a-4f7e-8ffc-49cb49305147 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.728933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.729644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.729929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.730205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.730961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:43.731809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.733367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131caae6-0701-48ef-ba2f-c7cf5f89ac91 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.748064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 322/632] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:43.749708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.750002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.750282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.759741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.771571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:43.773452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:43.810645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dc786-be9a-4b9d-bf2b-a6bc93ce07ad None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:43.812956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:43.856781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:43.857577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:43.857964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:43.858245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:43.859092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:43.859403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:43.860445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 323/633] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:43.860874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:43.861301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.886419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 311/634] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:43.894603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:43.991623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf35550-dba2-41cf-8fed-4a13fd53426c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.024639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c90930-9389-4299-8058-4f59a675fa73 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.034816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.036009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.036452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.037002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.049182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.052178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.054389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.061807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.062737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.063110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.063462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.074587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:44.076901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:44.077807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:44.078706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:44.078706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:44.078706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.078706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:44.078706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 324/635] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:44.079399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:44.083219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:44.085133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:44.086298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 312/636] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:44.092954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.114976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.206165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834f04-0aa8-4364-904a-a42139eb71d5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.220957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff626a-c7f6-40f6-8e3b-0d60af9ca9b1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.243385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.244259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.244545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.244892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.256987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.257663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.257947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.258224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.259347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:44.259663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.261391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.270788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 313/637] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:44.275857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.320386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.321179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.322991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e1f2a-1c6c-4a65-b268-d66ebb486464 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.342735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.380567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ca460-0e04-4276-a2ca-bd77aa1d76a8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.415977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.416717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.416935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.417207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.417925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:44.418227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.420152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 325/638] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea055f6-67ce-4c90-8871-bd0c8c4d0717 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea055f6-67ce-4c90-8871-bd0c8c4d0717 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea055f6-67ce-4c90-8871-bd0c8c4d0717 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 326/639] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:44.429809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea055f6-67ce-4c90-8871-bd0c8c4d0717 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.438535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 314/640] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:44.439217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.540338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea055f6-67ce-4c90-8871-bd0c8c4d0717 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.541931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845fd10-3c1d-4224-909b-1069dbbd32b2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.549847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.560082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.577552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.577800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.578337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.578569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.578684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.578845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.578940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.579122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.588949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.589211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.590414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.590670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4df47c87-fffc-4af5-a13e-942c253c1c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:44.608147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:44.609064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:44.610592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:44.610592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:44.610592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.610592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:44.610818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:44.611700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:44.613370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:44.613370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:44.613370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:44.613370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:44.618413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 327/641] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:44.618413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.618413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.618413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.619014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 315/642] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:44.619014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.631702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:44.632912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:44.705862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 328/643] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:44.706731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.733714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983109b2-7b72-4d09-9485-f9f211be0f5e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.769858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.770655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.770942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.771222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.779221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.780713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:44.795894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:44.796870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 316/644] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:44.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.817046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.817809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:44.824141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c095c-3c19-4a24-b154-ef8c8a65175f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.862888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.863557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.863843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.864204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.864879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:44.865192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.866599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.879788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 329/645] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:44.879788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.908026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b3a9c-0f70-49d3-984f-5587ee7d603f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:44.943189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:44.944185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:44.944678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:44.944964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:44.945752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:44.946063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:44.947445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:44.963554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 317/646] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:44.963554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:44.991821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51946b9-9817-4b0d-8c97-3e208d7b61ca None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.026579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.027363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.027653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.028479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.038577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:45.046756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:45.047909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:45.048961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:45.049622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.049695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 330/647] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:45.049695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.076270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.078065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f68b29-a177-4f30-9fd1-34ee5f8405c4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.116259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.116853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.117474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.117597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.118417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:45.118683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.120080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.143992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 318/648] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:45.143992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.161920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916031cd-5a54-440d-8aba-56e84c2684f0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.195974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.196791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.197078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.197353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.204929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.206379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:45.220336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:45.221335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 06:54:45.222292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:45.226086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 331/649] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:45.227020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.249615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b61f9a-b649-4a5c-a1b7-a53dab366f94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.284805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.285566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.285853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.286122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.289204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.295162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.297105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:45.314479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:45.315472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:45.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 319/650] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:45.316932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.323583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.347882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72708407-6f5e-4cd3-92b2-69ff8a8a9469 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.385408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.386224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.386514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.386797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.388088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:45.388432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.389803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.398243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 332/651] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:45.399032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.435868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c779b23-9975-4454-9d24-022d3a9e829c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.475700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.476088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.476399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.476680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.503559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b28dd-bf76-4a56-ae1a-cc1f3b6bc4c2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.539630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.540447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.540830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.541193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.541302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.542186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:45.542593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.544212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.549694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.551097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.560266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.568407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 333/652] 10.222.0.23 () {56 vars in 1137 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:45.568407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.661921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 320/653] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:45.662856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.675255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a07e3-cce1-4f91-94cd-448995decbac None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.718692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.719154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.719440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.719715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.720611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:45.720849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.722282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.735649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 334/654] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:45.737601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.773043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.781598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6503778-a219-4cf5-b0b0-566e2bf34a73 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.800002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:45.819261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.820143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.820433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.820707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.831519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.833009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.844149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adf0c8-dc4a-423d-b9b8-4e0c216ed876 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:45.851109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:45.852189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 321/655] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:45.857651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:45.881896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.882681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.883270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.883550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.950534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c78ccb-4e0b-417a-8957-7576dfed0651 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:45.951046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.952315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7170004-3952-4c66-8104-cd920ce9bc84 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:45.985200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:45.985990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:45.986269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:45.986542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:45.997020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:45.998474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.006840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.008296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:46.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:46.022761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:46.023699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:46.023699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:46.023699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:46.023699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:46.023699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 322/656] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:46.023987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.050472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 335/657] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:46.051699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0ae4d-b97b-4200-8515-f1c02970077c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.051984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0ae4d-b97b-4200-8515-f1c02970077c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.052286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0ae4d-b97b-4200-8515-f1c02970077c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.053525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 336/658] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:46.054203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0ae4d-b97b-4200-8515-f1c02970077c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.127729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefca64-c14a-4c9a-9b09-1c4ca86b02d3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.154766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0ae4d-b97b-4200-8515-f1c02970077c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.161449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.162092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.162373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.162649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.163470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.163781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.165250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.170872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 323/659] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.171627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.189649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.190334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.190618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.191021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.192095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.192436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.193862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.212073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 337/660] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.212073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.234181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.247267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.297993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff972d-5bbf-48ac-8611-993e79686fd7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.322051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2fc6-70dc-46f8-ac00-55d01750444e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.338269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.338947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.339236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.339519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.340335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.340649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.342034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a9b69-7f11-459a-a0ba-da3603623130 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.354748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 324/661] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.359390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.359390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.359390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.363003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.363788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.364438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.364438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.373879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:46.375090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:46.376510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.377953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:46.400843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:46.401801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:46.402721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:46.402721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:46.402721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:46.402721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:46.402721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 338/662] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:46.403001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.452976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 325/663] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:46.453822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.491046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.510615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.523913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3abafd9-1ed5-47e5-b94e-a5743ac68cec None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.557617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.558404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.558686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.559552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.559668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7615a-eed8-4650-a7ec-49217037a200 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.567555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.569027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:46.583744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:46.584987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 339/664] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:46.586357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.604505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.605300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.605585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.606425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.613737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.615196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:46.629597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:46.630701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 326/665] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:46.636803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.699838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed42a7b-c275-4e50-b831-e65019196587 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.730684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.737731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.738391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.738671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.738950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.739680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.739990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.741389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.753880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 340/666] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.756302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75d62b2-3355-473e-8d04-871b172ec250 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.757078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.771861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:46.794019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.794943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.795307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.795657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.806608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:46.814930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:46.816038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:46.817145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:46.823386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 327/667] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:46.823386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.878832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e4a16-1228-4658-af35-6a986f89d293 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.914080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.914863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.915143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.915445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.916374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.916688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.918231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.922057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f9873-2905-41eb-8f01-77bef0ce41c1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:46.941522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 341/668] 10.222.0.23 () {56 vars in 1139 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.941522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.961793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:46.962553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:46.962915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:46.963262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:46.964143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:46.964624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:46.966765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:46.978880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 328/669] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:46.979690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:46.994166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.017380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.048027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77674976-cdfc-47ac-97a8-996d1e655473 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.084556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.085338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.085620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.085891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.091499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b821fd1-8c25-46ca-ac6b-5a28c950141d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.093043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.094709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:47.108520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:47.109407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:47.110300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:47.112141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 342/670] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:47.113201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.131667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.132621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.133053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.133479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.134475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:47.135069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.136487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.150872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 329/671] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:47.150872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.218499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf60f33-9477-4b36-8acd-b3f8c97ca5d2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.228689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.249826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.255139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.255792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.256073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.256212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e4993-6311-4e65-a9af-0dfcd4f68080 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.256373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.293148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.294749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.295033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.295302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.296221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=468b120f57fe4807b969c390cbcabfbb) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:47.296536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.297966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.312148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 330/672] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/role_assignments?user.id=468b120f57fe4807b969c390cbcabfbb&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:47.312148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.343957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.345744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.447769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3746832-7a3f-4bff-9e94-9fd1de50297e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.477146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 343/673] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:47.478214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.491013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.491901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.493316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.493316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.493827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.494142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:47.494457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.496213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.511383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 331/674] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:47.513927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.546080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.608293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef945cda-289b-4f28-9214-4af51ddff623 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.646640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d43e-b88d-4eff-8f32-88db04599c53 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.647668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.648693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.649072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.649454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.660747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.662337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:47.681346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:47.682462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:47.683396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:47.683396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:47.683396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:47.683396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:47.683955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 344/675] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:47.688028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.692643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.693521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.693809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.694084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.714638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.716903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:47.735241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:47.736199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:47.737106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:47.737106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:47.737106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:47.737106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:47.740248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 332/676] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:47.741521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.776758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.820066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:47.829053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13ebfb-b741-41b0-a13b-2eb2a31b973b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.857724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedb4b-e257-46d9-bfb6-6517ee07616d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:47.872809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.873951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.874337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.874715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.875949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e318b1b6386c4a2d9e7f324e8fc2cfe6) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:47.876388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.877969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.885269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 345/677] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/role_assignments?user.id=e318b1b6386c4a2d9e7f324e8fc2cfe6&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:47.886662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:47.904614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:47.905642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:47.906093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:47.906530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:47.907414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:47.907904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:47.909587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:47.923897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 333/678] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:47.929756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.021913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20b307-2b5c-486a-918a-b60a876da1dd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.054478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.055778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951aa45-866b-4807-99fb-9c88e3df3a7c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.065768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.066800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.067273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.067733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.068807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:48.069312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.071004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.085625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 346/679] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:48.092312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.093472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.112794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.113939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.114429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.114861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.129764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.132993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:48.160725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:48.161959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:48.163181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 334/680] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:48.172179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.260391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bd753-ee45-4b4c-86f8-a29b8e1853a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.301674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.302392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.302682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.302958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.307093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29db26-ecc2-400b-a0d4-cf05a03a1d3e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.314466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.314466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-300b9875-7b49-4d19-9c51-c5af9b4003ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:48.328983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:48.330212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:48.331337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 347/681] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:48.331962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f720-afc3-432d-b8b2-5d8ffdd22693 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.331962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f720-afc3-432d-b8b2-5d8ffdd22693 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.332138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f720-afc3-432d-b8b2-5d8ffdd22693 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.333466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 348/682] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:48.338480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6f720-afc3-432d-b8b2-5d8ffdd22693 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.362439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.363120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.363406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.363685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.364560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:48.364874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.366307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac76085-6810-4388-be70-b927913a9920 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.386310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 335/683] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:48.391444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.391444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.391444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.405101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:48.406263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:48.411527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.457242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.482042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 336/684] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:48.483231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.486186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6f720-afc3-432d-b8b2-5d8ffdd22693 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.538705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987c39-0934-4d93-9037-4defce922615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.539628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987c39-0934-4d93-9037-4defce922615 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.539968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987c39-0934-4d93-9037-4defce922615 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.540284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de987c39-0934-4d93-9037-4defce922615 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.541127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de987c39-0934-4d93-9037-4defce922615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:48.541668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de987c39-0934-4d93-9037-4defce922615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.544267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de987c39-0934-4d93-9037-4defce922615 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.567883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 349/685] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:48.567883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987c39-0934-4d93-9037-4defce922615 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.604062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bb141-199d-4fba-abfe-69e13836601c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.640859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.641919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.642474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.642949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.694085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de987c39-0934-4d93-9037-4defce922615 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.708740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.718141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.744864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.746360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.747438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.748675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.749990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.750443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.764515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:48.772306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:48.773278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:48.774121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:48.778701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 350/686] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:48.779531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.876845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 337/687] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:48.877533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.902300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e3225-db86-45a4-9708-e516c08d2f54 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:48.948693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:48.950281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:48.951582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:48.952026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:48.953026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=014dfb1baeb24fa7ab38923a6962db4b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:48.953495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:48.955136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:48.962154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:48.963486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 351/688] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/role_assignments?user.id=014dfb1baeb24fa7ab38923a6962db4b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:48.964991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:48.984971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a04223-3e22-4ec9-846d-47360a428e28 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.009509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.025538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.026371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.026748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.027122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.028642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:49.029060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.030568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.039792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 338/689] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:49.046065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.107007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5b29b-183f-49ae-8cb0-029b02712566 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.155673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.157489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.157489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.158291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.160918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511a29c-2978-4d2b-a35a-011d207e3891 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.182620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.184735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:49.214363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:49.215909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:49.217182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:49.217182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:49.217182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:49.217182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:49.217182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 352/690] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:49.222245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.231846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.233900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.234372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.234851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.253631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.255928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.258721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:49.281329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:49.282309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:49.283308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:49.283308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:49.283308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:49.283308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:49.286168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 339/691] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:49.291252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.338862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.363700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca00057-83f9-4467-80a6-c426a078313a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.412488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.413434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.413900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.414357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.415273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:49.415777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.417337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.417446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da55ba6-bd74-4fcc-801a-3f0c2a05b55a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.431978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 353/692] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:49.441164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.453673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.454363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.454646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.454923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.455641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:49.455946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.457444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.471673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 340/693] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:49.476907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.565107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.601369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f042960-4a9c-4ce7-9fc3-087063424159 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.646202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1743a5-aa3e-48c5-9eb5-40f10d9348b6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.661315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.662152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.662435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.662760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.675176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.676656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:49.705463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:49.706441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:49.707341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 354/694] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:49.716869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.720989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.721783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.722063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.722340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.725634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.739123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.741876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:49.762913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:49.763915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:49.764897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 341/695] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:49.770105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.871325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7fdb8-a80e-447a-a1e5-dde77ec1f888 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.878959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9875d3-d208-4eef-8b10-2f68f05932fa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:49.900507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:49.907804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.908509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.908794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.909073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.909975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:49.910288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.911729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c8820-b9ea-49a2-b300-246b33b45042 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.916907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:49.917421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.917708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.917980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.918861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:49.919182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:49.920822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00301ef-fba9-40dc-a7a2-baf9792875fb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:49.925521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 355/696] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:49.927020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec859-3ace-4932-b45c-2a2368ecb764 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.927309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec859-3ace-4932-b45c-2a2368ecb764 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.927584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ec859-3ace-4932-b45c-2a2368ecb764 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.932989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 356/697] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:49.933836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec859-3ace-4932-b45c-2a2368ecb764 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:49.939896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 342/698] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:49.945438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:49.945739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:49.946025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:49.963405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:49.964608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:50.000406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.040223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 343/699] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:50.041103 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.043321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ec859-3ace-4932-b45c-2a2368ecb764 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.079976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.080918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.081376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.081808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.082772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=601afd44d1f54f939e87511ee463cc4a) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:50.083231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.084820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.091772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 357/700] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:49 2022] GET /identity/v3/role_assignments?user.id=601afd44d1f54f939e87511ee463cc4a&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:50.092931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.165098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15881a43-2b9f-4b3a-b2d9-73263c7d2b66 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.166700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.203665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.204286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.204570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.204842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.206280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d7b0d-18e7-48a3-a5d7-0660ee50426f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.234487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.245821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.247522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.248029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.248534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.249394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.250982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-32314762-4dcb-4064-a531-4fc4bcf73b20 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:50.266964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:50.268045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:50.269020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:50.272356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 358/701] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:50.274032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d552f4-0a4e-4773-9fbe-5e59d673eea1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.274536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d552f4-0a4e-4773-9fbe-5e59d673eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.275037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d552f4-0a4e-4773-9fbe-5e59d673eea1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.277019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 359/702] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:50.278284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d552f4-0a4e-4773-9fbe-5e59d673eea1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.289087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.290592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c79518-e60a-4263-8f24-9b2956862789 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.413106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d552f4-0a4e-4773-9fbe-5e59d673eea1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.414001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 344/703] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:50.415288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.415574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.415852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.428400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.444732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:50.446051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:50.452339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.453054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.453336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.453615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.454433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:50.454683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.456129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.461218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 360/704] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:50.461995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.523951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.531982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 345/705] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:50.532664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.586259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77054b3-84f0-45af-84e2-f7a750097785 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.625478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.626274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.626553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.626829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.638683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.640423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:50.658255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:50.659280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:50.660865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:50.660865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:50.660865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:50.660865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:50.664124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 361/706] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:50.664686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0586c11-2f45-4148-af76-2f19d4fca552 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.665062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.704970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.712891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.713838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.714223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.714599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.716006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:50.716444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.718831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.726863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 346/707] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:50.727921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.801676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c280af-fc0a-4ba5-8933-f846f0e2d674 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.813361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:50.841698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7c85c-047f-4a8d-b0c3-7f61debe3f8f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:50.846726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.849549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.849854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.850138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.876921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.879991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:50.911278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:50.912702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:50.913606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:50.913606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:50.913606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:50.913606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:50.917075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 362/708] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:50.920712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:50.921402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:50.921688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:50.921964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:50.922701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:50.923010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:50.924430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:50.925404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:50.940186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 347/709] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:50.945486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:51.040564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:51.046738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5a77b-3372-4db1-90a8-16559248e754 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:51.057899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cce740-bb4e-4eef-9ff2-c449ac39ccfd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:51.084197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:51.084919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.085204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.085482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.086304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:51.086618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:51.087986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:51.099941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:51.100894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.101279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.101554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 363/710] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:54:51.101671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.107134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:51.113114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:51.114834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:51.126167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:51.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:51.131941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:51.132922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:51.135384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 348/711] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:51.145517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:51.237073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52804a31-107f-4190-98fe-810dcf5bf608 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:51.303688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:51.304305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.304621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.304876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.305575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:51.307298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:51.342994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 364/712] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:54:51 2022] POST /identity/v3/users => generated 254 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:54:51.346569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7e70e-edca-4fba-aafa-fca02bba58c6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:51.365577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:51.387357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:51.388031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.388343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.388619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.389418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:51.389781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:51.391627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3c652-0d0d-4a56-8954-9d34b5f13520 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:51.405730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 349/713] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:51.413323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:51.530497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:51.574763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f593881-b940-4bd3-930f-bac64fc9453e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:51.617626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:51.618311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.618594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.618870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.619727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:51.620039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:51.622259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ea727-61ed-445e-a491-484df3b061ed None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:51.628900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 365/714] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:51 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:51.728409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fb7e-df32-44e1-99a6-0c1fcb740ce5 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.728749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fb7e-df32-44e1-99a6-0c1fcb740ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.728979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433fb7e-df32-44e1-99a6-0c1fcb740ce5 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.730411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 350/715] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:51.736463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:51.736779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:51.737034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:51.756771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:51.758000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:51.834033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 366/716] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:51.859945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fb7e-df32-44e1-99a6-0c1fcb740ce5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:51.969351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433fb7e-df32-44e1-99a6-0c1fcb740ce5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:52.007214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:52.008006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:52.008378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:52.008671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:52.009366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:52.010747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:52.025105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:52.026124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:52.026999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 351/717] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:51 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:52.032778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:52.114610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:52.149509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65b727-8baa-4b39-a955-168e519e53b8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:52.185116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:52.185779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:52.186063 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:52.186342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:52.187052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] RBAC: Adding query filter params (name=network) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:52.187356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:52.188750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03707e0e-c4ce-484d-a627-d837f29791ad None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:52.193740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 367/718] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:54:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:52.201360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:52.271302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:52.350839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac0ce-87d9-4fa8-a519-012933b748b1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:52.392903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:52.393485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:52.393768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:52.394045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:52.394755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] RBAC: Adding query filter params (type=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:52.395060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:52.396465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2e137-3546-483f-8935-58ad36dad52a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:52.404646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 352/719] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:54:52 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:52.945691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926c17-c3b1-47c0-bfde-525c5a764fe6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:52.946071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926c17-c3b1-47c0-bfde-525c5a764fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:52.946294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2926c17-c3b1-47c0-bfde-525c5a764fe6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:52.947934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 368/720] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:52.953645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b7b8-e7e0-4de2-b6c3-8e7a2c3ba4a8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:52.953941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b7b8-e7e0-4de2-b6c3-8e7a2c3ba4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:52.954227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d0b7b8-e7e0-4de2-b6c3-8e7a2c3ba4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:52.977154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-61d0b7b8-e7e0-4de2-b6c3-8e7a2c3ba4a8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:52.978269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-61d0b7b8-e7e0-4de2-b6c3-8e7a2c3ba4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:53.066083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 353/721] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:53.102688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2926c17-c3b1-47c0-bfde-525c5a764fe6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:53.252379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2926c17-c3b1-47c0-bfde-525c5a764fe6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:53.283625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03091d8-f317-44c1-bdf5-bccf63c358a9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.284278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03091d8-f317-44c1-bdf5-bccf63c358a9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.284764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03091d8-f317-44c1-bdf5-bccf63c358a9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.286622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 354/722] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:53.290941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:53.291788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.292187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.292555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.305886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:53.306734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.307021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.307299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.312213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:53.313405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:53.335354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:53.343232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:53.344199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:53.345068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:53.353453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 369/723] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:53.359402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:53.376298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:53.434417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 355/724] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:53.490885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:53.509105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127eccf5-517d-4bcb-b31f-be82e590c489 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:53.544009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:53.544714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.544999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.545287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.546043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:53.546348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:53.547720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:53.556495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 370/725] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:53.562384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:53.668659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:53.718426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a136e-534a-42e8-9a97-17a9651114b2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:53.751653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928149-f688-4344-9712-1c4bfda063c7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:53.794077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:53.794860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.795142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.795423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.817439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:53.818910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:53.822079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:53.822748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.823028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.823302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.824032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:53.825441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-78ce66cb-3b92-4699-bcca-1927e4b790d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:53.847302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:53.848294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:53.849201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:53.849201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:53.849201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:53.849201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:53.854475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 356/726] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:54:53 2022] POST /identity/v3/services => generated 251 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:54:53.856649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 371/727] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:53.863168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:53.917850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:53.975513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c62a9-a638-4e79-9b18-86b8bf4cb94d None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.975862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c62a9-a638-4e79-9b18-86b8bf4cb94d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.976075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c62a9-a638-4e79-9b18-86b8bf4cb94d None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:53.977750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 372/728] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:53.983577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f809e6-06f7-4ea0-a31e-8e23934dd55a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:53.983865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f809e6-06f7-4ea0-a31e-8e23934dd55a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:53.984207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f809e6-06f7-4ea0-a31e-8e23934dd55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.006805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d1f809e6-06f7-4ea0-a31e-8e23934dd55a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:54.008047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d1f809e6-06f7-4ea0-a31e-8e23934dd55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:54.059266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1630-37aa-484f-8d64-1b7f8dcfdfcf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.140901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.141662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.141951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.142223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.142820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 373/729] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:53 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:54.143317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:54.143317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:54.144715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:54.154631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeb7cf-2159-42db-acce-912ef613d896 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.154930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeb7cf-2159-42db-acce-912ef613d896 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.155212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeb7cf-2159-42db-acce-912ef613d896 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.156810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 374/730] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:54:54.158865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 357/731] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:54.161622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.161913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.162192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.163984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.180541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:54.181794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:54.212458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.268099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.270783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 375/732] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:54 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 06:54:54.271902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.305739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b680f67-d669-453a-a23d-93b65ca79357 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.347770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.349189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.350074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.350576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.361941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:54.364447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:54.384848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:54.386267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:54:54.387964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:54.389649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 358/733] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:54.392223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.410357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d223b-3f07-43f4-946e-c01a167e2fa2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.447195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.448045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.448472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.448793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.460957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:54.469251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:54.470208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:54.471049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:54.475483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 376/734] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:54.476414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.520464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.525708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023e00b-1f2d-40db-aaea-9cb9679b4e29 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.562767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.563952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.564467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.564943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.573828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.577835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:54.586665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:54.587655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:54:54.588543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:54.595479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db753739-7831-4d59-a32b-d38c17ceb272 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.596097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 359/735] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:54.597193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.643013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.643617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.643902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.644207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.646284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:54.646819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:54.649174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:54.670402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 377/736] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:54.671636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.731190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3c7c9-2a18-4602-84e4-0be34e3a9c6f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.767021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.767855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.768269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.768644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.769576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:54.769926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:54.771399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:54.781298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 360/737] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:54.782336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.792463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.810931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23719-5d9c-4ad5-a7ae-b410de2a01e5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.847476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.848319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.848822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.849200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.850046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:54.850451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:54.851963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:54.854231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:54.865502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 378/738] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:54.866366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:54.897786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3e8bb-8bea-4cca-8932-586fcefd874d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:54.935423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:54.936077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:54.936394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:54.936676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:54.975811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082de1f5-9840-4f3e-8e59-3abe8d229c86 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.013841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.014642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.014929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.015483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.019454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.020954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ab9a1-944a-4889-b3a6-6b022c248a51 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.034998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.036465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.048576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:55.060627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:55.061572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:55.062492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:55.062492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:55.062492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.062492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:55.062492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 379/739] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:55.063333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.089381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.134754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 361/740] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:54 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 06:54:55.136014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a610b-4428-454f-bd32-7bede4a661c2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.136352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a610b-4428-454f-bd32-7bede4a661c2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.137145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a610b-4428-454f-bd32-7bede4a661c2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.138590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 362/741] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:55.143275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a610b-4428-454f-bd32-7bede4a661c2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.177418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dbfe6-a6c1-48bd-8e96-32d4ec060798 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.213150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.214234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.214517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.215395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.226463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.227916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:55.245410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:55.246874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:55.247793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:55.247793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:55.247793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.247793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:55.247793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 380/742] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:55.248647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.259418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a610b-4428-454f-bd32-7bede4a661c2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.297793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.298570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.298853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.299126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.299484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.300581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:55.300875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.302243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd6b33e-23ef-4eb3-a9b7-1e104dd8dcce None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.318862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 363/743] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:55.318862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.318862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.318862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.334922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:55.336115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:55.352464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.356206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded46b6-b5c6-493b-bb92-99e1e0febc9a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.397737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.398539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.398830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.399112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.409006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 364/744] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 06:54:55.410361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.421533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.423063 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:55.442193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:55.443483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:55.444454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:55.444454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:55.444454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:55.444454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:55.448226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 381/745] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:55.449706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.525925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce840d9-1f8e-4f1d-9d6e-4781a7de99b5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.553702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac80202-4ad0-4538-9b6d-9c9d3a349fde None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.555232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.562377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.563052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.563339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.563615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.564380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:55.564690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.566074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.577820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 365/746] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:55.578614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.594673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.597414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.599265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.599602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.599883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.608208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.609888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:55.625218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:55.626252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 382/747] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:55.627211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.690606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a63a3b-b3f8-46c9-be41-5d4ad1804a52 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.727715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.728402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.728684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.728957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.729796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.731817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.738987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956ba29-1e0c-41d9-92e8-69650e034bc0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.756150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 366/748] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:54:55 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:54:55.756150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.809205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.810090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.810468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.810872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.811718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:55.812134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.813656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.827232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 383/749] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:55.828372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:55.850252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.868081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:55.916344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b79b34-23a6-4c43-9c84-944edfea30aa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.958943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be8207-be68-4315-b759-9c9154747f4d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:55.959894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:55.960744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:55.961780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:55.961780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:55.970162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:55.971653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:54:55.989656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:55.990880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 367/750] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:55.992184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.006017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.006701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.007415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.007718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.008605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:56.008920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.010351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.030180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 384/751] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:56.031014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.111137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d68ad-1d51-44f0-a35b-810d3a59fe85 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.119752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.142844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.145404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95571ff5-6017-4dac-93f7-428797015793 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.149488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.150282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.150567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.150846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.158719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.160276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:56.174956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:56.175852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:56.176821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 368/752] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:56.177967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.187585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.188295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.188582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.188863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.189698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:56.190009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.191433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93c2a9-b2a3-4a42-acd9-68a516e021f7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.209405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 385/753] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:56.210617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027241ee-0b4e-4479-bfd2-d8d378cc3b5e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.210917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027241ee-0b4e-4479-bfd2-d8d378cc3b5e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.211199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027241ee-0b4e-4479-bfd2-d8d378cc3b5e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.216148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 386/754] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:56.216148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027241ee-0b4e-4479-bfd2-d8d378cc3b5e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.301354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44859fb-bfb0-48c8-a23e-5f0647d8d5a6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.325543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027241ee-0b4e-4479-bfd2-d8d378cc3b5e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.339393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.340802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.341093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.341373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.352037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.353539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.361860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.362523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.362805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.363082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.363860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:56.364201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.365605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200134c-49d2-4ac6-ad9f-3dd030f3aa2f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.367319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:56.371281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:56.372415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 369/755] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:56.378055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.390824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 387/756] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:56.390824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.390824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.390824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.407028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:56.408259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:56.430644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.481254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 388/757] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:54:56.482123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.500215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741bfa-6441-4f5d-81ad-9ad8c02f0727 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.541020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.542254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.543316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.544038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.606867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651711c-16c7-412e-9e5a-19152bb5c111 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.615431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.616930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.634490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.647401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.648296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.648755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.649184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.650161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:56.650685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.652123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.661144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.669713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 389/758] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:56.669713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.733811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 370/759] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:54:56.734508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.788335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacccca2-cdf9-4bbb-9afb-6cd99de4b8d9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.823349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.824024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.824340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.824620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.825440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=601afd44d1f54f939e87511ee463cc4a) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:56.825750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.827166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.840155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 390/760] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/role_assignments?user.id=601afd44d1f54f939e87511ee463cc4a&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:56.840155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.840459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e66e0f6-7cc7-4aba-8515-66c828d5b442 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:56.882590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:56.883421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:56.883708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:56.883985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:56.884693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:56.887464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:56.897615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:56.905384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:56.906427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:56.907277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 371/761] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:56.908185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:56.922756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:56.966694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b1d51-a87d-4211-acf3-ab8f4a0cfe5f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.014501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.015612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.015943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.016272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.023719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.025225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:57.041525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:57.042478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:54:57.043497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 391/762] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:57.043996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.071790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe592e8-ef7c-4fbe-91ea-86f370402d3b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.113951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.114750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.115035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.115309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.129032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.130838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:57.153241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:57.154235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:57.155157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:57.155157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:57.155157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:57.155157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:57.161588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960f75-5333-4d03-99f4-881adf2fa8cc None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.161741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 372/763] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:57.161741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.171905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.201048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.202628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.203120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.203657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.204600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.205119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.206810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.212444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 392/764] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.213770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.226822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.270335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c114efa-5dc8-42cd-82ce-7a712c7c7ecd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.310112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.310326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.310615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.310795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.311829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.312410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.316523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.329291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 373/765] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.330200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.335415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79cdf5-807c-4feb-8ad9-0f910226245b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.380820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.382171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.383176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.383891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.384852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.385559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.387784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.409570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 393/766] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.410705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.447551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501dab7-4d84-4c2c-8631-304338cdfa13 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.455457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.471847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.488555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.489362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.489717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.490160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.491054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.491432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.493273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.503811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 374/767] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.504872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.549757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aab8c6-c07b-4698-9f33-7da0573d9731 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.599853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.601492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.601965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.602426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.605820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=e19cf623e22c485fb1bb8f0ada7114b9, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.607957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.618448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f00f77-eda3-4f0a-bf28-504c20e096b9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.625669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 394/768] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:54:57 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:54:57.626729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.685274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.686298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.686725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.687099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.704608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.706264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:54:57.732658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:57.733647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:54:57.734559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:57.740183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 375/769] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:57.749253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.749446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.761181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c3b0e-8795-4271-a6a0-82599e96f5ee None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.806671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.807273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.807568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.807848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.808752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] RBAC: Adding query filter params (interface=public, service_id=37c4f3b259a74b2d896570b9fbd0c0f7, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.809067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.811183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.820229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 395/770] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/endpoints?service_id=37c4f3b259a74b2d896570b9fbd0c0f7&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.829832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:57.864610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:57.931814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae735e-de34-41fa-87b7-20fadd215d05 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.977189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:57.977573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c41879-988c-4006-bcb7-d27141e9f2c3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:57.977849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.978130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.978408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.979196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:57.979503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:57.980990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ae3f3-8bc7-490e-97e9-0d139a9ea4c3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:57.993707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 376/771] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:57.995056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f68d-7944-44bb-a360-19e4d4a40a75 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:57.995339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f68d-7944-44bb-a360-19e4d4a40a75 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:57.995611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328f68d-7944-44bb-a360-19e4d4a40a75 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:57.997146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 377/772] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:58.002439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.002993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.003485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.022642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:58.022825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.023451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.023739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.024074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.024794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.025163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:58.026172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.036205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 396/773] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:54:57 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:58.037609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:58.086564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:58.165803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 378/774] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:54:58.195468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:58.239093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:58.274293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf447df-da8c-4273-a357-0ec5ec483b8d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:58.310856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.311876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.312397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.312880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.313901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=c5eecb100c404edba01e417cc4baa10b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:58.314433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.316562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.322679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53666b-a5de-4f6c-8c5b-13758fef6b1f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:58.322932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 397/775] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/role_assignments?user.id=c5eecb100c404edba01e417cc4baa10b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:58.362460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.363193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.363480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.363761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] PATH_INFO: `/v3/services/network` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.364481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.365869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:58.379016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:58.380059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:58.380959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 379/776] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:54:58.387392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:58.495559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:58.525940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2f740-d9e4-4970-846c-305fddb0a4a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:58.563049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.564023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.564513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.565001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.565897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] RBAC: Adding query filter params (name=network) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:58.566387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.567930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.578150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 398/777] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:58.579684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:58.697756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c105f9-8231-41b9-b8e1-169c67d8c847 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:58.704130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:58.746853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.747527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.747812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.748091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.752909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] RBAC: Adding query filter params (type=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:58.753230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.754624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.764238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 380/778] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:58.766444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:58.891609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83629b69-2585-4d71-8cfb-7079c30d8a7d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:58.935811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:58.936510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:58.936788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:58.937078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:58.937886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] RBAC: Adding query filter params (interface=public, service_id=fa584192c4e446f88c2a30db3afb13fc, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:58.938212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:58.939642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe136d-757d-4c84-91db-0a5d64c71d6d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:58.951350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 399/779] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/endpoints?service_id=fa584192c4e446f88c2a30db3afb13fc&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:58.952844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:59.026365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01c86-6ba4-4980-ab2d-4159954a76c1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.026693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01c86-6ba4-4980-ab2d-4159954a76c1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.026983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b01c86-6ba4-4980-ab2d-4159954a76c1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.030579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 400/780] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:54:59.035035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.035333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.035901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.067567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:54:59.068790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:54:59.082406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:59.096035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d517b91-b144-499f-8518-a97cf6114a7a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:59.171352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:59.173307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.173886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.177455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.178364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:59.179936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:59.189469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 381/781] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:54:58 2022] GET /identity/v3/services => generated 772 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:59.203462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 401/782] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:54:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:54:59.238127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:59.288004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:59.367998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd171-3aa6-4fa9-8042-8d6cdeb74e27 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:59.407977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:59.409110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.409588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.410036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.421037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:54:59.428878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:59.429816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:54:59.430668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:59.433076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 382/783] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:54:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:59.439650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:59.584029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42608d-6ec6-4dcf-b786-6784f43c1226 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:59.627564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:59.628555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.629050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.629507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.630488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:54:59.631017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:59.636695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:59.646299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 402/784] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:54:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:54:59.652094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:59.776541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:54:59.802899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0a42-b852-4e43-a9cc-574739d83013 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:54:59.848944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:54:59.849660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:54:59.849948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:54:59.850227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:54:59.863940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:54:59.865426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:54:59.890250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:54:59.891197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:54:59.892130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:54:59.892130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:54:59.892130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:54:59.892130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:54:59.892130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 383/785] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:54:59.897336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:54:59.901340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.027680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2150-6e9f-4ce2-9a31-185b978abd74 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.062508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.063507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.063995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.064531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.065448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:00.065978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.067544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b3cb6-2c4d-4f63-9677-989acd9a0d04 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.080448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 403/786] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:00.088138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.240987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ef09d-7f02-40ae-a2ef-3b5dbdb94b8a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.280557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e727-5eff-4e3d-86e3-6c260362cdaf None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.280866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e727-5eff-4e3d-86e3-6c260362cdaf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.281118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e727-5eff-4e3d-86e3-6c260362cdaf None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.282915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 404/787] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:00.287746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.288035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.288358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.290975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.291871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.292184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.293098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.305353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:00.306555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:00.309791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.311373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.313355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:00.331082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:00.332368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:00.333366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 384/788] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:00.337938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.393802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.414356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 405/789] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:00.451835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.491787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c08539-91f5-4d82-b8d7-6e237413af53 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.530459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.531444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.531917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.532414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.533563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:00.534081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.535785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.560425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 385/790] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:00.561947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.569285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd65d7-530e-4baf-b940-3ed649993786 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.611382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.612518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.612986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.613464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.627701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:00.637890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:00.638826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:00.639710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:00.642844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 406/791] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:00.648559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.663239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.713135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.739032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9d18-1f3b-41b7-96f0-d61169784132 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.766491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b3927-6706-49d8-9799-d1e19f672a88 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.777821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.778857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.779355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.779885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.791409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.793463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.801183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.801942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.802459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.802938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.803907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:00.804460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.806038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2835d9-9d28-45da-bca3-92c51f0fc6df None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:00.814178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:00.815174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:00.816151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:00.816151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:00.816151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:00.816151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:00.822575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 407/793] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:00.822575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa7560-38a1-4553-a6a1-971d8bf5d2e4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.822575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa7560-38a1-4553-a6a1-971d8bf5d2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.822575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa7560-38a1-4553-a6a1-971d8bf5d2e4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.823043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 386/793] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:00.824493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 408/794] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:00.824859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.826264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa7560-38a1-4553-a6a1-971d8bf5d2e4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:00.928583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c93015-bbe0-4e68-81ca-9417dc4b613c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.930427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa7560-38a1-4553-a6a1-971d8bf5d2e4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:00.939228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.947823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:00.966781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.967580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.967863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.968158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.970252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:00.970917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.971198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.971474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:00.972231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:00.972542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.973900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28289838-1770-4a26-84d0-d2a658630e18 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.979411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:00.981931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:00.990768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 409/795] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:00.990768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:00.990768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:00.990768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-51e65318-222a-4b55-a994-05355f25ac4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:01.004426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:01.005360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:01.006404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:01.006404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:01.006404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:01.006404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:01.006834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:01.006908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 387/796] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:01.007740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1e38d-1026-41bb-aa09-8fc69e2789fb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.008090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1e38d-1026-41bb-aa09-8fc69e2789fb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.008216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:01.008445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d1e38d-1026-41bb-aa09-8fc69e2789fb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.017375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 388/797] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:01.017375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1e38d-1026-41bb-aa09-8fc69e2789fb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.081985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 410/798] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:01.083187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.140275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d1e38d-1026-41bb-aa09-8fc69e2789fb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.177619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.178595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.179033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.179455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.187076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.187205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cfc9a-833f-41a9-b163-d60ffd592f38 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.188822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.195199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-91919cbe-0c36-47f3-b008-c0036bf43622 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:01.203794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:01.204920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:01.206014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 389/799] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:01.206614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.206614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.206772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.226330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:01.227027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.227683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:01.227792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.228100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.228491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.229281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:01.229650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.231671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.243148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.247926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 411/800] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:01.248830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.301311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 390/801] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:01.302129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.387466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633aea5-7f4a-4462-9446-5180d70d4d0b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.424348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.424930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ebd6cf-b616-450a-ab78-2586b41f5f02 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.425760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.426425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.426907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.428510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.430070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:01.442591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:01.444140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 06:55:01.445276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:01.447896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 412/802] 10.222.0.23 () {56 vars in 1129 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:01.448967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.465106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.466018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.466769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.467056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.468173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:01.468493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.470561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.481868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.486978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 391/803] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:01.487637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.502287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.565398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8f826-e03f-46e8-ab29-0e338d47740b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.595998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41544e-4aa4-466d-a6a5-5d3cd1dc8fdf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.601665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.602695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.603203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.603585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.612080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.613664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:01.631141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:01.632142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:01.633113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:01.633647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.633647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.633647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.633647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.639289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 413/804] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:01.640237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.645475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:01.653217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:01.654172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:01.655019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 392/805] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:01.656333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.739903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.751050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:01.751549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bdc0c-c837-4e09-ad1f-42c5f189c717 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.786358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.787107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.787468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.787813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.788635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:01.789019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.790433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.793426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fd97e-ff7b-48fd-9192-744b05a82e35 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.801147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 414/806] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:01.801147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.829696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.830505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.830785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.831063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/601afd44d1f54f939e87511ee463cc4a/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.833065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=601afd44d1f54f939e87511ee463cc4a, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.834509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.851077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 393/807] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:01 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/601afd44d1f54f939e87511ee463cc4a/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:55:01.852400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.918125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d170faa-a97e-4cc6-a5ab-3493fa43ace2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.957378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:01.958067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:01.958349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:01.958628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:01.959476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:01.959790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:01.961277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:01.981484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 415/808] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:01.982639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:01.995037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e721b-e72b-440f-b3ea-5edceb0f8df7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:01.999338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.035933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.043351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.044244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.044531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.045047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.046562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:02.047226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.049559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666ae86-1d6a-42e0-a698-4b953ed8c5f5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.061577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 394/809] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:02.065961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25962ad-26bb-437d-9a9d-d947ec56ca84 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.065961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25962ad-26bb-437d-9a9d-d947ec56ca84 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.065961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25962ad-26bb-437d-9a9d-d947ec56ca84 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.065961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 395/810] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:02.066799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25962ad-26bb-437d-9a9d-d947ec56ca84 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.107619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d9656-4989-433e-a17a-7a2788fe7bd9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.143670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.145145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.145434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.145712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.146424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:02.146735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.148102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.156262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 416/811] 10.222.0.23 () {56 vars in 1124 bytes} [Mon Jun 20 06:55:01 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:02.157082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.182856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25962ad-26bb-437d-9a9d-d947ec56ca84 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.219551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.220232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.220521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.220795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.238851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.265300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60967b8-24b3-4c91-bb15-9f0f912f3ca0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.281877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.294187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.296451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.299889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.300712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.300998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.301273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.312830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.314278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-fb69a8a1-81d9-4689-91f3-a2d8f5e1758c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:02.337913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:02.339080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:02.340376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:02.340376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:02.340376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:02.340376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:02.344029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 417/812] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:02.345169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.345456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.345738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.364581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:02.365828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:02.407808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 396/813] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:02.408499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.442599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 418/814] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:02.443549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.507189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.523863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.533291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecded7bc-41f5-4477-9d49-5b401e68ccc7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.552979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2cd7be-bead-471c-b506-57d099782408 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.571157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.571819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.572117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.572394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.573138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.575139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.591792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.592482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.592767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.593041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.593761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:02.594068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.595460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.598089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 397/815] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:02.608687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 419/816] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:02.609180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.610053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.756472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a3a78-fb3e-4857-bc5d-a78c148ab23b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.776721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.778604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb766-5226-48a5-a904-7f036addd3b1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.802082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.802936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.803218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.803493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.804172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.806105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:02.818827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:02.819847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 06:55:02.820762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:02.822086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 420/817] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:02.822366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.827548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.828648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.829152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.829652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:02.841492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:02.843192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:02.846433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:02.863519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:02.864510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:02.865410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:02.865410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:02.865410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:02.865410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:02.867638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 398/818] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:02.873263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:02.952354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903f20d-9fa5-4a45-86f4-626bcd2285c4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.991043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ef5f2-315e-4663-bf36-5ac041effbbb None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:02.994762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:02.995934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:02.996445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:02.996751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.004878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.006506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:03.022594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:03.023734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:03.025160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:03.029320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 421/819] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:03.030244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.034344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.035025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.035315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.035591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.036340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] RBAC: Adding query filter params (name=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.036651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.038090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.043565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 399/820] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.048811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.099195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.112829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.157653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266378b3-ebd1-4016-9d02-172664b73081 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.181909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23092f-55f3-427b-aa01-349324473bdd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.193754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.194543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.194933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.195406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.196514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.196845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.198785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.220786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 400/821] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.220786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.230522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.231677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.232137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.232838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.233710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.234177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.236230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.252822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 422/822] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.253889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.329522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377c508-8086-4f05-a280-6a1b618562fd None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.347966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.359984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640387-791f-4520-ac9b-9af9c659ca4d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.361529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.367411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.368256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.368617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.368970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.369781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] RBAC: Adding query filter params (type=network) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.370160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.372190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.382653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 401/823] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.383520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.400638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.401751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.402258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.402746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.413728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.415471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:03.434815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:03.435764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:03.436706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:03.436706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:03.436706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:03.436706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:03.443164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 423/824] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:03.444249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.492399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29cad-725e-4175-a2d4-fbc4c0f5e86c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.530029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.530859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.531131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.531407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.538495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.539943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.552429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613e48-7a92-49d5-b0cd-cbc24941920c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:03.554511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:03.555405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:03.556333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:03.560634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 402/825] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:03.561551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.592522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.593537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.594020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.594505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.595446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.597098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.609688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.636780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 424/826] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:03 2022] POST /identity/v3/endpoints => generated 327 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:03.639526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.671883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.734384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c5d51-ff2d-4e97-894d-c76a67c74dc9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.813708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.815147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.817402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.817954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.819831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.820387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.822000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.837641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 403/827] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.845166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.864846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8be0-0a0a-466c-864c-1797ffa66484 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:03.931611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:03.932400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.932757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.933140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.934007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:03.934389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:03.935896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becc94-9b47-4efa-96dd-398266c2340c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:03.954472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 425/828] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:03.955787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520c22-e482-4931-aff6-ee32ae2749ba None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:03.956558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520c22-e482-4931-aff6-ee32ae2749ba None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:03.956851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520c22-e482-4931-aff6-ee32ae2749ba None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:03.958220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 426/829] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:03.960712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520c22-e482-4931-aff6-ee32ae2749ba None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:03.983471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:03.988234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19b7178-ae93-4042-8971-c18732e206c3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.025419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.026751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.027563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.027815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.036402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.038009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.043291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-719f31b6-b9e3-4ce3-acdd-9e37e65796bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:04.051862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:04.052970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:04.053954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 404/830] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:04.054441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.054606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.054855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.074287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:04.075450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:04.085445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520c22-e482-4931-aff6-ee32ae2749ba None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.120994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.122042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.122332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.122619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.123812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:04.124155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.125930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.140148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 427/831] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:03 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:04.140148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.161352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 405/832] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:04.184815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.279397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2815116-21d6-4b9f-8505-1551548d7ca6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.291015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:04.307084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad59013-363e-4437-9068-e6a0502679f3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.326912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.326912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.327175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.327360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.328922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:04.328922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.331348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.332064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:04.347475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 428/833] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:04.349358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.350243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.350594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.350934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.353191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.362676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:04.370348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:04.371818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:04.372730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:04.376854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 406/834] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:04.378049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.475835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701af32c-4b90-4e34-a217-a3da614feb09 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.489038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5979-ca26-4a3f-a694-357dac6d04ac None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.514115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.515255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.515536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.515905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/c5eecb100c404edba01e417cc4baa10b/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.524554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.525224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.525511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.525795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.526555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:04.526896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.528315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.532858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=c5eecb100c404edba01e417cc4baa10b, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.534397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.537563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 407/835] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:04.544221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.552467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 429/836] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:04 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/c5eecb100c404edba01e417cc4baa10b/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:55:04.563147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:04.666114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc5a43-c976-4be6-acd2-f04ea044b947 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:04.716148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:04.716148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.716148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.716148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.723838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:04.725391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-a1628074-bdcd-4c21-a699-11fcb642cff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:04.744327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:04.746142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:04.747636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:04.747636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:04.747636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:04.747636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:04.747636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 408/837] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:04.752233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.781538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f966f0-d8fd-4374-b98b-8b46654535e7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.781831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f966f0-d8fd-4374-b98b-8b46654535e7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.782112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f966f0-d8fd-4374-b98b-8b46654535e7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.783485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 409/838] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:04.789891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:04.791233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:04.791735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:04.811146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:04.812769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:04.878983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:04.893804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 410/839] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:04 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:04.916928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:04.946192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.017598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024db87-6481-4356-90ec-c06e011ef42a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.059024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.059690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.059964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.060245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.060974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:05.061274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.064091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.083048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 430/840] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:05.084637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.097314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa87b7-4c97-4771-9fbc-b0d49f9c3472 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.145678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.147064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.147898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.148412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.149289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.151622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:05.166221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:05.167553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:05.168489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:05.173739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 411/841] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:05.175056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.214193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c432c20-78df-42ab-9e1c-2f0422c4cd99 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.217303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.233593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.259414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.260422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.260721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.261690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.270481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.271945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.286218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cbcda-1a24-436e-bad7-ebd92ad2edcf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:05.288079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:05.289023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 06:55:05.289978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:05.294780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 431/842] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:05.296096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.333266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.335171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.335683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.336188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.337105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:05.337630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.340311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.346835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 412/843] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:05.352261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.410855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bb156-8b52-4594-8e7d-f742b6f7aebf None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.449777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.450335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.450630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.450982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.451889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:05.452282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.453783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.463721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fcb66-872f-4e2a-b69c-203d332bd7da None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.477820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 432/844] 10.222.0.23 () {56 vars in 1135 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:05.479209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.485877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.500728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.501662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.502141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.502699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.503719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:05.504247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.505802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.506251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.513266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 413/845] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:05.600835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3ecea-0736-461f-9360-ddede4d14e18 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.639162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.640093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.640407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.640689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:05.713147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:05.715197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:05.740999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:05.826020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 433/846] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:05.828173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:05.952920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aaf77-66bd-4d4d-998f-475d4e7fbe56 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:05.992476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:05.993694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:05.994155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:05.994605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.127805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.129297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:06.147374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:06.148332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:06.149245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:06.149245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:06.149245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:06.149245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:06.149245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 414/847] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:06.157198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.294782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111ecd4-04fc-42db-87ec-d0cdd4c183c1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:06.328988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:06.330836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:06.331664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.331951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.332281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.333061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:06.333377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.334799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814726b-1408-464d-9d5f-6d0a05e40461 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.364234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 434/848] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:06.369691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.412197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:06.463671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8bef6-f3b5-4499-856a-59b53590bac8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.464547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8bef6-f3b5-4499-856a-59b53590bac8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.464798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8bef6-f3b5-4499-856a-59b53590bac8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.466316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 435/849] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:06.477465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.477754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.478036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.489355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cd58e-6294-46c2-b6bf-72343a3014e1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:06.517907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:06.520247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:06.555269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:06.556803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.557567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.558077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.567620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.569357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:06.590731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:06.591681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:06.592762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:06.594013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 415/850] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:06.601678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.624137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 436/851] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:06.642715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.738026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:06.745662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d8d87-43fa-428a-8310-1ef8b8dbeb09 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:06.774916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12769b86-a685-475d-be39-809cc33bd5f2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:06.788794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:06.790794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:06.791443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.791810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.792056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.792830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:06.793139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.794744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee189b-c2bc-469f-9d3c-5748cc55586c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.813097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 416/852] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:06.813097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c316a6-6f7f-4f58-88f0-7daa6349eb76 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.813097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c316a6-6f7f-4f58-88f0-7daa6349eb76 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.813097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c316a6-6f7f-4f58-88f0-7daa6349eb76 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.813097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 417/853] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:06.816545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c316a6-6f7f-4f58-88f0-7daa6349eb76 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.817433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:06.818240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.818521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.818792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.819670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.821239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-49942954-d190-4f8e-8d1e-c29dba664679 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:06.833959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:06.834902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:06.835775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 437/854] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:06.837347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.837347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.837347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.853269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:06.854432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:06.921588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 438/855] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:06.922412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:06.940708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c316a6-6f7f-4f58-88f0-7daa6349eb76 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:06.976158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:06.976812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:06.977096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:06.977374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:06.978196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=601afd44d1f54f939e87511ee463cc4a) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:06.978508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:06.980495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:06.986192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 418/856] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/role_assignments?user.id=601afd44d1f54f939e87511ee463cc4a&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:06.986946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.024174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.071193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022aa5-80d3-4f0c-95d7-2a6c8b756754 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.082034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.119696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.120964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.121329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.121632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.122360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:07.122669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.124068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.134080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 439/857] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:07.135124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.136496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095368f-a959-493b-a163-40ac22f7f4ed None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.187380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.188356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.188674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.188960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.202896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:07.210577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:07.211553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:07.212524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 419/858] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:55:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:07.218023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.265799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a5b7a-a363-4704-bcb2-bd2d3c07e69b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.316468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.318611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.319682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.320014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.320373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.322099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:07.322356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.324232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.331412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 440/859] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:07 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:07.344974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245675f1-1652-495e-83ac-82d9ceb29731 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.386678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.387449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.387742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.388024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.388888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:07.389205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.390984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.401246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 420/860] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:07.409273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.449207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.526723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26af7c9-00e8-457e-a20b-1d8827947f31 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.563764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.564621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.564914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.565195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.576558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.578000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4ed9513c-44c0-4f9c-bf8b-6dcde5bbd389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:07.595949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:07.598026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:07.600141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:07.600141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:07.600141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:07.600141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:07.600141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 441/861] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:07.602874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.687706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ca54-2f68-4195-9469-bd4de86978ec None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.688244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ca54-2f68-4195-9469-bd4de86978ec None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.688354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8589ca54-2f68-4195-9469-bd4de86978ec None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.689742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 442/862] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:07.697527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.698116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.698617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.711955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad2c7e7-a843-45a0-97a8-ea6124832973 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.724579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:07.726429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:07.751786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.752584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.752937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.753287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.754084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:07.754462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.755906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.769939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 421/863] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:07.777156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.780587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.821814 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 443/864] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:07.842775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:07.861396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:07.911624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef73f7-8c36-46a6-8dfb-656581d9f279 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.958993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:07.960815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:07.961582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:07.962075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:07.963373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9983f8f3-bcd6-4e24-aced-a4c18a9a234a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:07.971532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:07.973698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:07.995062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:07.996689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:07.997862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:08.004813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:08.004813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:08.004813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:08.005324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:08.005475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 422/865] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:55:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:08.006273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:08.006563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:08.007968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:08.028213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 444/866] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:07 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:08.087554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:08.134382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238b482-35e1-4a3c-9cf6-de5a331fc5d8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:08.180442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:08.181349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:08.181639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:08.181976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:08.182866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:08.183181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:08.184725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:08.209520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 423/867] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:55:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:08.216457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:08.297767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:08.321498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59658a-6f72-4835-8faf-e56af4fb6d5c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:08.356694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:08.357632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:08.357923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:08.358205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:08.427643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:08.429029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:08.507347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:08.533508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 445/868] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:08 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:08.539130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:08.645202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002390ff-ad7b-47b5-9596-dfc48430328c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:08.666138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:08.685243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:08.685958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:08.686242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:08.686521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:08.697161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:08.699458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:08.717615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:08.718497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:08.719392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:08.719392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:08.719392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:08.719392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:08.719392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 424/869] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:08.725657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:08.827403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ab9d4-b04e-4ba0-8961-ef206f2c2b93 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:08.861478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:08.862522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:08.863005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:08.863511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:08.864447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:08.864950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:08.866493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:08.879126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 446/870] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:08.884832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:08.990126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e8d83b-f4b8-405e-9889-800588e74966 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.028065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.028881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.029165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.029434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.036471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.037915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-9cc4f5c5-1556-4eab-b1cb-c54fefad8b9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:09.051571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:09.052502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:09.053387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 425/871] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:09.060534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.097038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:09.138584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e05f2-36fd-47a7-aacd-181c2a9c1a3d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.138934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e05f2-36fd-47a7-aacd-181c2a9c1a3d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.139115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e05f2-36fd-47a7-aacd-181c2a9c1a3d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.141054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 426/872] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:09.146154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.146441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.146722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.163639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:09.164857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:09.175986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5af5-3ebb-419d-8f40-156a154cac87 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.189857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:09.210549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.211349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.212006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.212006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.212962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:09.213339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.214922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.227492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 447/873] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:09.233451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.238904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 427/874] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:09.261521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.350937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:09.357890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284ace-11e2-45c5-a1a3-39b1c734d28e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.376235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c4a5e4-1e89-42a1-9d5e-7c848f729588 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.395960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.396879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.397263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.397640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.398574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=c5eecb100c404edba01e417cc4baa10b) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:09.399009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.400642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.407564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 448/875] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:09 2022] GET /identity/v3/role_assignments?user.id=c5eecb100c404edba01e417cc4baa10b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:09.423123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.423969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.424412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.424761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.425509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.427003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-08b02b02-7533-4594-ac9a-ec187596fc70 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:09.440786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:09.441866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:09.443163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 428/876] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:09.451522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.494002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5798-cd90-47c0-8be6-1eca323f18cf None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.494320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5798-cd90-47c0-8be6-1eca323f18cf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.494590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc5798-cd90-47c0-8be6-1eca323f18cf None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.500079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 429/877] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:09.501897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.502192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.502477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.520879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:09.522121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:09.541565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:09.573487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c41dc-e18d-4807-b067-55caa0f61e17 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.603605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 430/878] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:09 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:09.615868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.616938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.616938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.617107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.617792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:09.618100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.619459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.627947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 449/879] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:09.629099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.637437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:09.667712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:09.765584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275a30d-29f0-4c3c-81d8-ee4150712443 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.772420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b706ba-18b7-4b2e-a724-5b73c52b1454 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:09.808613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.809282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.809570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.809849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.810566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:09.810900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.812279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e75c2-868f-4858-b1d3-08d3d0226a7a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.813330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:09.814901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:09.815667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:09.816189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:09.817120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:09.818685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e08506-b923-48b1-af47-a2b9bf37a953 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:09.822784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 450/880] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:09.837779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 431/881] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:09 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:10.319618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadd41f-7ee5-4c45-9a7a-991c5dd83da9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:10.320364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadd41f-7ee5-4c45-9a7a-991c5dd83da9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:10.320877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadd41f-7ee5-4c45-9a7a-991c5dd83da9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:10.322744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 451/882] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:10.328617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:10.329457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:10.329956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:10.349087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:10.350703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:10.360429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:10.430754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 432/883] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:10 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:10.455169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadd41f-7ee5-4c45-9a7a-991c5dd83da9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:10.550279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:10.565314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadd41f-7ee5-4c45-9a7a-991c5dd83da9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:10.600718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:10.601488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:10.601779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:10.602053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:10.602711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:10.604083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:10.616939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:10.617940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:10.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 452/884] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:10.625003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:10.730997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ca7f6-c0f3-43f7-90fc-d7388722bd91 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:10.766733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:10.767395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:10.767673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:10.767945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:10.768722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:10.769028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:10.770433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:10.775449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 433/885] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:10.780729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:10.882991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083b0cc-25d3-4624-ab10-70433544a20d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:10.917326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:10.917999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:10.918285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:10.918560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:10.919269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:10.919575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:10.920973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:10.926192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 453/886] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:10.931848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:10.937295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:11.053705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2a1e8b-442e-480f-ade1-e1cf7b610178 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:11.091328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:11.091989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.092305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.092584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.093375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] RBAC: Adding query filter params (interface=public, service_id=bbb8766d608b4c208c8fcbb22c79cad1, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:11.093690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:11.095061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b67c68-7539-481b-9dd4-cf110953427c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:11.102998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 434/887] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:55:10 2022] GET /identity/v3/endpoints?service_id=bbb8766d608b4c208c8fcbb22c79cad1&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:11.108181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:11.127835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:11.214071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c5045-c5c2-4f4a-a87b-8639b372388f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:11.248520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:11.249184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.249470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.249748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.250431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:11.251795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fe91af-8fe1-43dd-8d12-0ba1abeeeabc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:11.257849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 454/888] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:11 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:55:11.660658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18dbb2-60a6-4a8d-bfba-9e9b5a7566c6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.661203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18dbb2-60a6-4a8d-bfba-9e9b5a7566c6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.661203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18dbb2-60a6-4a8d-bfba-9e9b5a7566c6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.662801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 435/889] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:11.668519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2379a-1b4c-4a6e-9fab-70647f7d5f14 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.669062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2379a-1b4c-4a6e-9fab-70647f7d5f14 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.669547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2379a-1b4c-4a6e-9fab-70647f7d5f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.687623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-91b2379a-1b4c-4a6e-9fab-70647f7d5f14 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:11.689169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-91b2379a-1b4c-4a6e-9fab-70647f7d5f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:11.767717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 455/890] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:11.791507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18dbb2-60a6-4a8d-bfba-9e9b5a7566c6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:11.861185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:11.903639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18dbb2-60a6-4a8d-bfba-9e9b5a7566c6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:11.942695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:11.943428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.943712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.943992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.947727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d604cf-488a-4754-8f70-f521e046404e None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.948667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d604cf-488a-4754-8f70-f521e046404e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.949146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d604cf-488a-4754-8f70-f521e046404e None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.951245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 456/891] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:11.956301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f75f-1d4e-400c-b58f-32b251422eed None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:11.956974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f75f-1d4e-400c-b58f-32b251422eed None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:11.957448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c9f75f-1d4e-400c-b58f-32b251422eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:11.957652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:11.967213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:11.968557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:11.969446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 436/892] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:11.974080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:11.980801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-67c9f75f-1d4e-400c-b58f-32b251422eed None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:11.982247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-67c9f75f-1d4e-400c-b58f-32b251422eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:11.984972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.060170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 457/893] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:11 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:12.060170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3ec7-15fb-4542-b758-cefe60a5567a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.060170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3ec7-15fb-4542-b758-cefe60a5567a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.060170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3ec7-15fb-4542-b758-cefe60a5567a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.060955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 458/894] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:12.066358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.066641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.066915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.090278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:12.091468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:12.104925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45335c27-9524-488e-abba-04a61b1e1b0e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.133506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.145368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.146078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.146359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.147112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.148180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:12.148544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.150447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.167310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 459/895] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:12.167310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.167570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 437/896] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:12.167570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.238613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.270150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae116dc5-dad8-4398-b729-7c713e10b748 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.279508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64f784-8daf-486b-b8c4-e8d0cd2b0156 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.303914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.304751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.305032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.305309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.305954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.307318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.315390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.316201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.316484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.316756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:12.319819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:12.321180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:12.322056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 460/897] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:12.322906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.330904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.332388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:12.351307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:12.352436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 438/898] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:12.353349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.405599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.446197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60460-5eb0-40ca-abf9-89e0d4a38b3d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.467067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a0e14-88f6-45ec-8d6a-f4747ffde26e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.482647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.483312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.483713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.483996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.484864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.486235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.488429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.508251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 461/899] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:12 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:12.508251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.515824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.516691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.517083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.517464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.518403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:12.518822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.520584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.531948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 439/900] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:12.533216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.645225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63c992-5728-450b-b1dd-b959b96ffd7c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.660719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.668408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b591db-58e1-4d6a-8f9d-8ab545142880 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.692090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.693108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.693397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.693676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.696479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:12.696479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.696479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.711341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 462/901] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:12.717481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.730247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.731205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.731496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.731776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.732693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:12.733008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.734779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.741637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 440/902] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:12.748415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.777981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:12.825788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef5143-c522-4013-9d1b-3ecd8caa2983 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.863077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.864477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.865017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.865512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.872005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f9f7-0c64-4cb8-bc7a-9541dbc5f639 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:12.874233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.876025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:12.892496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:12.893480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:12.894436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:12.901712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 463/903] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:12.902898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.909807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:12.910481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:12.910767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:12.911061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:12.911899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] RBAC: Adding query filter params (interface=public, service_id=423499088cfd456bbc678d49f140dec7, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:12.912250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:12.914115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:12.924186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 441/904] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/endpoints?service_id=423499088cfd456bbc678d49f140dec7&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:12.927241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:12.965714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.035020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882baa05-ad6f-4d26-9156-1bb198fb1acd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.048215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9da807-76b8-4c7a-9996-e82e70278e1d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.071033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.076438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.077743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.078174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.078872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.079849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:13.080440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.082656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0373c73-2c94-4202-878f-89ae8578cde9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.095804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.096506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.096797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.097079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.097776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.099156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.105587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 442/905] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:55:13.110727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 464/906] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:55:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:13.113277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.225657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937694ae-ba1e-4778-a570-98fe9c0b4542 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.227790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.266189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.267119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.267593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.268037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.339187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352a0cd-0846-4253-96a2-e38e57c5cb8a None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.339369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352a0cd-0846-4253-96a2-e38e57c5cb8a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.339651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352a0cd-0846-4253-96a2-e38e57c5cb8a None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.340695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.341206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 465/907] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:13.342249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.346290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.346574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.346851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.365224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:13.366405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:13.459703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.461504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 466/908] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:13.472896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 443/909] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:13.478627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.487108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.598838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab535741-07d5-40cd-9aa9-b096040cb126 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.601165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.626455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5346f4-72e3-4c56-ac4c-387c6f997d46 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.654391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.655335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.655980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.656400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.664749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.666088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.666535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.667568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.668453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.669465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.670302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.670953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:13.684009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:13.685143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 06:55:13.686002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:13.688835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 444/910] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:13.695190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:13.696085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:13.697335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:13.697335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:13.697335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:13.697335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:13.700379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 467/911] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:13.702036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.703080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.780020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:13.832899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba8a906-e0f8-4f79-8043-7f1edd49b7c5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.837866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca193f-1f09-4a8e-9ebd-a6eb3a8c068e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:13.874626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.875793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.876313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.876783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.876943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:13.877647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:13.877991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:13.878252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:13.878371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:13.878843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.879177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:13.879545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:13.880915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.881046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:13.891557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 445/912] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:13.899329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 468/913] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:13.900673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.901673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:13.919331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:14.011094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5241e9-4c95-412e-a94a-4bcb1241bdc8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.034510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b54c9-8459-4fba-91aa-7de76bc8cf65 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.054915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.056298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.056724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.057159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.058064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:14.067811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.070217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.074066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.074831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.075265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.075641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.076515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:14.076919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.078365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:14.086983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:14.088073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:14.089833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:14.090291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 446/914] 10.222.0.23 () {56 vars in 1114 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:14.090291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:14.092233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 469/915] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:14.098200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:14.196801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:14.202870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407b3e01-9297-45bd-b4c6-2312243179cf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.229127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb000e70-a1ec-4eba-b5bb-0b84f56aeafd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.241353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.242438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.242932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.243415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.244565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.246885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.272367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.273090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.273376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.273648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.274547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:14.274793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 447/916] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:14.281642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.283124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6360a2f5-ddea-4eb1-85c8-89081d397be3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.302645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 470/917] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:14.309563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:14.359281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:14.431641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f5904-83ec-49c8-88cf-4350b648461c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.469378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.469950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.470229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.470501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.471321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] RBAC: Adding query filter params (role.id=32ed5330175c4f55a7f3b7b320dcbafa, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=c5eecb100c404edba01e417cc4baa10b) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:14.471629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.473070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc3e1e-e89f-45c4-a175-ce4fcd9ba959 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.479873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 448/918] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:14 2022] GET /identity/v3/role_assignments?user.id=c5eecb100c404edba01e417cc4baa10b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=32ed5330175c4f55a7f3b7b320dcbafa => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:14.678971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6614-31fb-4839-abb3-e025c97b0731 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.679369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6614-31fb-4839-abb3-e025c97b0731 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.679548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e6614-31fb-4839-abb3-e025c97b0731 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.681168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 471/919] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:14.687060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.687511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.687739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.707702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:14.708941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:14.785472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 449/920] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:14.808616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6614-31fb-4839-abb3-e025c97b0731 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:14.918752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e6614-31fb-4839-abb3-e025c97b0731 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:14.954341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:14.955138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:14.955421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:14.955696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:14.956386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:14.957745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:14.959649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:14.972493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:14.973613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:14.974497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 472/921] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:14.981219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.014047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:15.090150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4edc-d2e2-4c2e-816f-c682018638ac None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.126469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.127135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.127416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.127694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.128432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:15.128735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.130101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e01a6-87bf-4ebc-a874-9bc097235e86 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.135373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 450/922] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:15.145219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.255299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cfaf40-d6f5-4aca-8efd-7d12d965b5b1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.261102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5da2-7c86-4c2c-b04d-8f16b62af759 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.261390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5da2-7c86-4c2c-b04d-8f16b62af759 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.261667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b5da2-7c86-4c2c-b04d-8f16b62af759 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.263801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 451/923] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:15.271845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.272282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.272900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.294773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.295151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:15.295479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.295764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.296036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.296786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:15.296917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:15.297084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.298445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.304257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 473/924] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:15.310198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.375301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:15.390190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 452/925] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:15 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:15.412643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.430366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf804-bd26-49c2-a798-1a1b399bdf9b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.443711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:15.470288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.471381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.471889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.472565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.473720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] RBAC: Adding query filter params (interface=public, service_id=04727d1dd6a6417883e3cefe8d37f572, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:15.474310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.476047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.483265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 474/926] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/endpoints?service_id=04727d1dd6a6417883e3cefe8d37f572&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:15.492158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.542806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c648a-0df6-4af2-bfc9-e559772e865a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.588024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.589620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.589908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.590184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.590848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.592956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.598778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac8ffb-2840-46ee-a317-97b9136efd3d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:15.609246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:15.610250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:15.611108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 453/927] 10.222.0.23 () {56 vars in 1132 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:15.616363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.631745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:15.642875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.643593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.643881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.644186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.644909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.646586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.660617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 475/928] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:55:15.727433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:15.740698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb7dbf-648c-4236-8a7c-c7abfa043b6e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.782906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.783600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.783888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.784199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.784970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:15.785283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.786672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.793202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 454/929] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:15.798687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:15.911404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afed8c4-5a9a-499d-b3e6-201e4bac764d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:15.946668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:15.947693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:15.948742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:15.949235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:15.950144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:15.950674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:15.952303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42c45f-9373-488c-ace6-fdfcddc0a563 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:15.958210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 476/930] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:15.965634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:16.077629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc1084-caed-4cd2-a11d-5794f45737ae None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:16.114959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:16.115993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.116531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.117027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.117985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:16.119620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e4f50-720f-4c78-831c-97015845181b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:16.143178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 455/931] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:16.425404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d3b3-fdad-433a-a1dc-6828372c314b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.426067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d3b3-fdad-433a-a1dc-6828372c314b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.426067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea2d3b3-fdad-433a-a1dc-6828372c314b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.427560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 477/932] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:16.432880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd6d-cf0d-421b-9544-e2e1cc2ea184 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.433169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd6d-cf0d-421b-9544-e2e1cc2ea184 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.433449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd6d-cf0d-421b-9544-e2e1cc2ea184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.450579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5dbadd6d-cf0d-421b-9544-e2e1cc2ea184 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:16.451754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5dbadd6d-cf0d-421b-9544-e2e1cc2ea184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:16.453878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:16.527283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 456/933] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:16.550708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d3b3-fdad-433a-a1dc-6828372c314b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:16.616085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:16.639976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57ea42-0d37-4285-9faf-399c518892d1 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.640348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57ea42-0d37-4285-9faf-399c518892d1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.640577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57ea42-0d37-4285-9faf-399c518892d1 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.642194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 457/934] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:16.647091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.647377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.647651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.664047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:16.665297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:16.670556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea2d3b3-fdad-433a-a1dc-6828372c314b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:16.721290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:16.722105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.722494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.722868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.723758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:16.726578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:16.730150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING keystone.server.flask.common [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 20 06:55:16.756437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 458/935] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:16 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:16.758373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.sql.core [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 06:55:16.758373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 06:55:16.758373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [parameters: {'id': '510060a68aca444b9f48be80d1746c83', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 06:55:16.758373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80136) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 510060a68aca444b9f48be80d1746c83.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 510060a68aca444b9f48be80d1746c83. Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:55:16.767357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 20 06:55:16.768187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application yield resource Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise self.value Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application session.commit() Jun 20 06:55:16.769216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.session.flush() Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application compat.raise_( Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 06:55:16.770168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 06:55:16.771165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application result.read() Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:55:16.771975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [parameters: {'id': '510060a68aca444b9f48be80d1746c83', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.774783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:16.776909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 510060a68aca444b9f48be80d1746c83. Jun 20 06:55:16.779361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.781763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:16.784289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 478/936] 10.222.0.23 () {60 vars in 1140 bytes} [Mon Jun 20 06:55:16 2022] POST /identity/v3/projects => generated 262 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:16.790925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:16.855156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:16.900341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942e965-df9e-4be3-987c-cf8822402631 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:16.905173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d701b24-5e98-44ca-b4da-7c44a22dd9fa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:16.916269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:16.942658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:16.943447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.943702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:16.943836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.944012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.945191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:16.945775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:16.946253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:16.951743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:16.953439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:16.958636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:16.968276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:16.969170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:16.970068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:16.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:16.972716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:16.973563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:16.975187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 479/937] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:16.976006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:16.976794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 459/938] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:16.985848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.096011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3ca32-368a-4089-b6e1-17964a3b6840 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.111734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063979c0-6e81-4757-a182-39003d6f2af5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.138624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.140048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.140576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.141135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.141788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:17.142873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:17.143571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.145531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.154836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.155615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.155996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.156394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.157255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:17.157659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.159110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.175418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 460/939] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:17.176883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.177047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 480/940] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:16 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:17.299327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbfd7-6e23-4b73-8b3b-61b268832c65 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.339833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.340667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.340950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.341223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.354250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.355730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:17.375706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:17.376727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:17.377645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:17.377645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:17.377645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:17.377645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:17.377645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 461/941] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:17.383266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.417950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:17.493459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecbdca-c54f-44f7-b22c-b9e69e74ebb9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.529496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.530155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.530484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.530761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.531548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:17.531852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.533301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.545353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 481/942] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:17.550825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.593567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:17.658733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e942e5-f7f9-41cc-a34b-dc3962b7f99a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.694600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.695457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.695744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.696017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.706402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.707861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bed4eba3-76d1-494c-a46f-535f54a0cdc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:17.726532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:17.727540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:17.728542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 462/943] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:17.735314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.788311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1bd24-2987-4b3d-9257-58776f006944 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.788831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1bd24-2987-4b3d-9257-58776f006944 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.788919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee1bd24-2987-4b3d-9257-58776f006944 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.790468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 463/944] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:17.795439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.795726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.796007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.814394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:17.815506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:17.840133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:17.850146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad64955-0ba5-458b-8f1d-a1f7a21f13e8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:17.887623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:17.888325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:17.888614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:17.888896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:17.889983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:17.890293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:17.891712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:17.892415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 464/945] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:17.910200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 482/946] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:17.911262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.915968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:17.947397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.047706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f189ab-4647-40b9-bf75-4aa962e35531 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.056298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6910-9688-4d44-ba57-10a6f794e142 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.092604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.093342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.093629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.093911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.094189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.095220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.095703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.096233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.097089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.098662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.106506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.107946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:18.112739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:18.113710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:18.114569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.116638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.122673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 465/947] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:18.124480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:18.129456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:18.130344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:18.131265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:18.131265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:18.131265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:18.131265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:18.131265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 483/948] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:18.137284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.249510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.260436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208d77a-28a8-4b98-b0f0-39914fe76c9b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.261857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea828400-35b9-4865-9541-b9df46443648 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.296614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.297280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.297569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.297850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.298569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:18.298883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.300292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbc32-3703-44b0-b387-298d267eb976 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.304385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.305003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 466/949] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:18.305420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.305875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.306200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbce9b-7277-4d5d-8f4a-f928357588c2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.306486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbce9b-7277-4d5d-8f4a-f928357588c2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.306709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.306808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbce9b-7277-4d5d-8f4a-f928357588c2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.308323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:18.312675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 467/950] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:18.312675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbce9b-7277-4d5d-8f4a-f928357588c2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.312876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.314017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fefe0-c77c-4128-8993-278c0415e677 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.325864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 484/951] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:18.327589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.327882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.328184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.345340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:18.346894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:18.407631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.420137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 485/952] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:18.422523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbce9b-7277-4d5d-8f4a-f928357588c2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.433028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.466009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.466759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.467046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.467319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.468026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:18.468360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.469811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.474954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 468/953] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:18.477034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.501684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.541254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae943648-2252-4da9-a83f-27e6d9d7618b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.577967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.578918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.579530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.579813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.585264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac4e14-3eae-419b-ba52-822b4b4fdab5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.587205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.588718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:18.602379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:18.603237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 486/954] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:18.604924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.626647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.627715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.628240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.628694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.629543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.631093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:18.644146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:18.645237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:18.646115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:18.653637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 469/955] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:18.654342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.659631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.726476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc2ee-2bdf-41e3-b3f1-57df56ec62f3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.747059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.761805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.762697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.763184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.763461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.764226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.765829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.765934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f81fe4-e8d9-4611-b308-0d9813bd4ce2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.784443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 487/956] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:18.785586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.826693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.827433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.827681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.827885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.829073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:18.829370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.831427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.856533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 470/957] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:18.862110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:18.933745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd551f-c320-4872-bf92-b70c0bc459a5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:18.942365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:18.983744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:18.985494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:18.985494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:18.985494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:18.986626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:18.986626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:18.988481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:18.994355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 488/958] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:19.000411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:19.020521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ded537-dea1-4744-89d0-32c31aae2355 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:19.069237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:19.070420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.070711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.071332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/c5eecb100c404edba01e417cc4baa10b/roles/32ed5330175c4f55a7f3b7b320dcbafa` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.075872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=c5eecb100c404edba01e417cc4baa10b, role_id=32ed5330175c4f55a7f3b7b320dcbafa)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:19.077349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07167bb-4afc-41dd-a3a5-b7ada82ecdf7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:19.089584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:19.107110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 471/959] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:18 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/c5eecb100c404edba01e417cc4baa10b/roles/32ed5330175c4f55a7f3b7b320dcbafa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:55:19.131903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b31f35-4a92-4b6b-9ff4-b6a89177187a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:19.174415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:19.175103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.175384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.175658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.176412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:19.176720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:19.178265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f4690-8417-448e-a882-0ff0a73f521e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:19.190100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 489/960] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:18 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:19.480068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f204f1-8a99-4b9f-9b75-fe2b25140b19 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.480283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f204f1-8a99-4b9f-9b75-fe2b25140b19 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.480575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f204f1-8a99-4b9f-9b75-fe2b25140b19 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.482199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 472/961] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:19.487866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.488184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.488523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.506513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:19.507711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:19.538247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:19.584018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 490/962] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:19.608846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f204f1-8a99-4b9f-9b75-fe2b25140b19 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:19.719245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f204f1-8a99-4b9f-9b75-fe2b25140b19 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:19.755814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:19.756466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.756752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.757096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.757849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:19.759275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:19.760670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING keystone.server.flask.common [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 20 06:55:19.775368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.sql.core [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 06:55:19.775368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 06:55:19.775368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [parameters: {'user_id': '51335d8638124cdb976a4a5e7683f500', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 06:55:19.775368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80137) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result.read() Jun 20 06:55:19.793091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:55:19.794012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application yield resource Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application next(self.gen) Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise self.value Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application session.commit() Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 06:55:19.795007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.session.flush() Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application compat.raise_( Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 06:55:19.795944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 06:55:19.796905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application conn.query(q) Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application result.read() Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 06:55:19.797887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [parameters: {'user_id': '51335d8638124cdb976a4a5e7683f500', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:19.798772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 20 06:55:19.799590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:19.806314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 473/963] 10.222.0.23 () {60 vars in 1131 bytes} [Mon Jun 20 06:55:19 2022] POST /identity/v3/users => generated 160 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:19.807801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:19.903344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914604d-be23-4b92-a83e-46cda77d64a6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:19.938076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:19.938865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:19.939147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:19.939420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:19.946696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:19.949030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-35763798-f1e0-43fb-bf94-343c0b325d84 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:19.964215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:19.966456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:19.967665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 491/964] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:19.971341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:20.047603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:20.094643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038c1c8-bc28-461a-bbe4-41b8b6ce55c9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:20.129918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:20.130592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:20.130880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:20.131161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:20.131968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:20.132310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:20.133948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232028d-f044-4e5e-b2f0-b7c7d3d6ec8e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:20.173523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 474/965] 10.222.0.23 () {56 vars in 1101 bytes} [Mon Jun 20 06:55:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:20.974774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e93040-ab20-4949-8eae-563d7d9e70f4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:20.975262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e93040-ab20-4949-8eae-563d7d9e70f4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:20.975262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e93040-ab20-4949-8eae-563d7d9e70f4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:20.976793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 492/966] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:20.982547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794b56-457d-4273-90b2-363ec8e24586 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:20.983033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794b56-457d-4273-90b2-363ec8e24586 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:20.983113 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794b56-457d-4273-90b2-363ec8e24586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.000360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c6794b56-457d-4273-90b2-363ec8e24586 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:21.001875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6794b56-457d-4273-90b2-363ec8e24586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:21.069151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 475/967] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:21.092098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e93040-ab20-4949-8eae-563d7d9e70f4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.118442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.198567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e93040-ab20-4949-8eae-563d7d9e70f4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.217641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f305e0-2281-4dbd-83c3-e12feba97312 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.217921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f305e0-2281-4dbd-83c3-e12feba97312 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.218201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f305e0-2281-4dbd-83c3-e12feba97312 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.219730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 476/968] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:21.224574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd974-0d33-4b56-8fd2-4a66cffc7984 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.225206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd974-0d33-4b56-8fd2-4a66cffc7984 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.225464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bd974-0d33-4b56-8fd2-4a66cffc7984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.233897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.234699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.234983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.235258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.235908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.237727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.242644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b78bd974-0d33-4b56-8fd2-4a66cffc7984 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:21.243828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b78bd974-0d33-4b56-8fd2-4a66cffc7984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:21.250278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:21.251283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:21.252160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:21.256348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 493/969] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:21.257158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.296840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.319030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 477/970] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:21.320415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677211-4de7-4fca-a04b-bc53895b7bf4 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.320716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677211-4de7-4fca-a04b-bc53895b7bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.320981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84677211-4de7-4fca-a04b-bc53895b7bf4 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.324464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 478/971] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:21.327746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.328247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.329218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.347979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:21.349176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:21.349660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.362286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7460d-96ae-4d50-9387-338c97e0c885 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.396289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.396946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.397232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.397517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.398225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:21.398533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.399900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.404730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 494/972] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:21.405788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.419925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 479/973] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:21.420954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.521036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dcb12-992f-457a-8b6a-e5e5c7a21f79 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.524433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3a04b-37a9-470e-ab1e-474ca40c8e6e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.535568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.554498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.555299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.555582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.555861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.560044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.560739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.561027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.561311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.562021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:21.562336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.563710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.565737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:21.568615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 480/974] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:21.569424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:21.573887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:21.574929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 495/975] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:21.575872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.591584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.673975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d92b77-e084-46f1-8be0-8a1f378aab3e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.674443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656ad51-bc8d-42ce-9923-02f3d8b356e3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.709691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66879257-7644-4e72-901f-5d66d50195be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.710356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66879257-7644-4e72-901f-5d66d50195be None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.710641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66879257-7644-4e72-901f-5d66d50195be None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.710907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.711014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66879257-7644-4e72-901f-5d66d50195be None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.711562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.711670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66879257-7644-4e72-901f-5d66d50195be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.711850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.712157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.712966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] RBAC: Adding query filter params (interface=admin, service_id=04727d1dd6a6417883e3cefe8d37f572, region_id=RegionOne) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:21.713068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66879257-7644-4e72-901f-5d66d50195be None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.715860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.717331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.722938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 496/976] 10.222.0.23 () {56 vars in 1249 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/endpoints?service_id=04727d1dd6a6417883e3cefe8d37f572&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:21.723726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.744158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 481/977] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:21 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:21.744158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66879257-7644-4e72-901f-5d66d50195be None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.775825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.823564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:21.847939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4064783-91fd-4b45-8a55-6f42a7d721b3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.866477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66879257-7644-4e72-901f-5d66d50195be None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:21.889512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.890179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.890900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.891184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.892225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:21.892545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.894377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.904014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 497/978] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:21.906925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:21.907696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:21.908090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:21.909263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:21.909263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:21.909770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:21.910841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:21.920954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 482/979] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/services => generated 1727 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:55:22.028794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82e9d3-7589-42c1-9469-ed378752f1ca None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:22.064097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:22.064930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:22.065214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:22.065490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:22.074547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:22.080810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:22.082271 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:22.101081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:22.102107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:22.103033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:22.103033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:22.103033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:22.103033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:22.103033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 498/980] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:22.108048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:22.203810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:22.210848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91037a42-68ea-4ebf-8320-8aa78296db54 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:22.245788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:22.246450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:22.246733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:22.247008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:22.247722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:22.248026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:22.249410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:22.263219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 483/981] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:22.267897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:22.367082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b3cb9-e4da-4cc0-8786-efa36cac7d98 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:22.402091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:22.402962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:22.403242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:22.403512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:22.411544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:22.413075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:22.428174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:22.429170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 06:55:22.430142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:22.432429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 499/982] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:55:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:22.434976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:22.536187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94779beb-afe9-4d8e-8dc1-e4343a5cd50f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:22.570055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:22.570990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:22.571473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:22.571938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:22.573498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:22.573999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:22.575577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:22.597874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 484/983] 10.222.0.23 () {56 vars in 1149 bytes} [Mon Jun 20 06:55:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:22.598742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:22.650346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:22.702339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8da2c-0080-453a-bb7e-f30d3e97b91e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:22.737618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:22.738258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:22.738540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:22.738848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:22.805735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:22.807190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:22.854955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:22.909388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 500/984] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:22 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:22.914653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:22.929227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:23.019951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5ef509-71dd-458d-a7a9-e2e1073e731c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:23.055017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:23.055812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.056095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.056404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.067499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:23.068954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:23.086597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:23.087538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:23.088536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:23.088536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:23.088536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:23.088536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:23.088536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 485/985] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:23.093278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:23.191325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090e2f-9146-4d5d-8083-c1fa55502ce7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:23.224037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:23.224718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.224997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.225269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.225982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:23.226283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:23.227640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:23.243136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 501/986] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:23.245260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:23.307429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:23.346214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72855965-2217-44c7-8eeb-bc95696fabf1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:23.380100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:23.380918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.381216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.381485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.388121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:23.389829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:23.404215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:23.405179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:23.406108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 486/987] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:23.412768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:23.426643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:23.518497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3411da1-79eb-451c-946a-8871f27c0f1b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:23.551703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:23.552389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.552679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.552958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.553744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:23.554056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:23.561144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da01f61-0adc-4075-b3ff-625cac4385a7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:23.579491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 502/988] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:23.581101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:23.683645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b677fb-37a3-4035-a6cb-1eea82d020a5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:23.709152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:23.720674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:23.721336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.721623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.721902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.722747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] RBAC: Adding query filter params (role.id=32ed5330175c4f55a7f3b7b320dcbafa, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=c5eecb100c404edba01e417cc4baa10b) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:23.723060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:23.724522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c998e-a455-4017-92fd-1dd78bc554b2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:23.740208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 487/989] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:23 2022] GET /identity/v3/role_assignments?user.id=c5eecb100c404edba01e417cc4baa10b&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=32ed5330175c4f55a7f3b7b320dcbafa => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:23.803002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c95a-40cc-4cc6-91aa-fcc1df98f2ef None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.803202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c95a-40cc-4cc6-91aa-fcc1df98f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.803477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269c95a-40cc-4cc6-91aa-fcc1df98f2ef None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.804921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 503/990] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:23.810537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62285-b882-4329-8482-b72cf4a5b385 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.810835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62285-b882-4329-8482-b72cf4a5b385 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.811122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc62285-b882-4329-8482-b72cf4a5b385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.828621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cfc62285-b882-4329-8482-b72cf4a5b385 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:23.829833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cfc62285-b882-4329-8482-b72cf4a5b385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:23.905969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 488/991] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:23.928083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c95a-40cc-4cc6-91aa-fcc1df98f2ef None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:23.981404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdb8b2-02f3-416f-a831-fad4b917d458 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.981694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdb8b2-02f3-416f-a831-fad4b917d458 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.981978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cdb8b2-02f3-416f-a831-fad4b917d458 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:23.983470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 489/992] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:23.988101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:23.988421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:23.988703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.005952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:24.007138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:24.007725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.042681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269c95a-40cc-4cc6-91aa-fcc1df98f2ef None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.078284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.078474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 490/993] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:23 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:24.078994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.079282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.079563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.080256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.081632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:24.094005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:24.100311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.101157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 504/994] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:24.101958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.103517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.154891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.210308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d02a0b-6867-488c-9b75-a7b2c4ee1161 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.210704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3472317-2666-45c9-86db-7baab823ec21 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.247739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.248702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.248856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.249314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.249314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.249519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.249774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.250054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.250216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.250764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:24.251074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.251836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.252486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.253690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.262849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 491/995] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:24.264037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:24.266231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:24.267213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:24.268072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 505/996] 10.222.0.23 () {56 vars in 1126 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:24.272989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.370827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ee987-3700-46cb-8c8c-8896a5c33dd4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.382872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f516b2-b806-4093-9393-7b997774ddf2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.386425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.409813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.410476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.410757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.411034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.411736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:24.412041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.413414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.418283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 492/997] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:24.423837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.424335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.424994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.425275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.425549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.426319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:24.426624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.428059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.433236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 506/998] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:24.438464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.505520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.533694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26d047-eebd-40d5-9250-dd754a61ac50 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.546299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf051aa-8d0b-4c9d-8856-1af6ec8ee1c7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.570423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.571084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.571375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.571659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.573032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] RBAC: Adding query filter params (interface=public, service_id=d55b6cbddf094b86aaa4ddfe66889ce4, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:24.573351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.574721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.580391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 493/999] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/endpoints?service_id=d55b6cbddf094b86aaa4ddfe66889ce4&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:24.585301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.586149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.586874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.587222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.587562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.588388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:24.588698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.590040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.595020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 507/1000] 10.222.0.23 () {56 vars in 1122 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:24.600201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:24.628867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.690923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c198460-2eeb-439f-a3ae-984d428fb6b8 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.709438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcd22a-18da-4542-bb14-6ae1efc29b0f None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:24.728886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.729546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.729873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.730566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.731568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.733147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614d1cc-13f5-405f-9777-e5e5eceaa15d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.735553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:24.740943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 494/1001] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:24 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:55:24.756004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:24.756749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:24.757041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:24.757332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:24.758111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:24.759529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c915973-1563-4c21-92c6-d416492c5f95 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:24.779417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 508/1002] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:24 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:25.825525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be23f0-040f-4a11-be4b-01ea0a0550ca None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:25.826033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be23f0-040f-4a11-be4b-01ea0a0550ca None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:25.826153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be23f0-040f-4a11-be4b-01ea0a0550ca None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:25.827587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 495/1003] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:25.832606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:25.832979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:25.833174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:25.850300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:25.851480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:25.930170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 509/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:25 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:25.952957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be23f0-040f-4a11-be4b-01ea0a0550ca None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:26.059596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be23f0-040f-4a11-be4b-01ea0a0550ca None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:26.061920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:26.095096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:26.095879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.096179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.096457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] PATH_INFO: `/v3/services/image` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.097109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:26.098454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:26.111157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:26.112256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:26.118333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 496/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Jun 20 06:55:25 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:26.121357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:26.224430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb85af-0267-4b40-94bb-83d0334efc49 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:26.238367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:26.260924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:26.261581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.261863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.262140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.262840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] RBAC: Adding query filter params (name=image) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:26.263142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:26.264792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3775a-9e05-4fcb-a93f-dfbf8afff92e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:26.269865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 510/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:26.275861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:26.376816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be002e-51fc-4d10-b82a-c64b8ea00d27 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:26.411329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:26.411992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.412302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.412581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.413286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] RBAC: Adding query filter params (type=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:26.413587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:26.414942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7b763f-a9e9-4fe7-b2ac-c0a2af0be591 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:26.429407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 497/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:26.825045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a23fe-474e-423a-937e-efd2eb52edbb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.825421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a23fe-474e-423a-937e-efd2eb52edbb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.825624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a23fe-474e-423a-937e-efd2eb52edbb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.826991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 511/1008] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:26.832177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.832631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.832796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.849735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:26.850916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:26.861009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240754a-2259-4dc2-a633-60fa0f8b37b3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.861472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240754a-2259-4dc2-a633-60fa0f8b37b3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.861648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240754a-2259-4dc2-a633-60fa0f8b37b3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.863095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 512/1009] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:26.867689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:26.867979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:26.868286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:26.880289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:26.886038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:26.887241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:26.921489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 498/1010] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:26.944950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:26.954453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 513/1011] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:26 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:26.978215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:27.047180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8f09b-556b-4fd9-8d9a-00c80768a746 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.076784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af6f8f-45c4-4b83-a124-68a176ccfc79 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.083049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.083831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.084172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.084641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:27.084948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.085617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.088378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:27.103251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:27.104397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:27.110519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 499/1012] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:27.111356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:27.112204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.113314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.113599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.113884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.115068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.116532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.118163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 514/1013] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:27.123675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:27.144393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:27.225646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7b6f0-01e1-47e2-8c92-9683434261b0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.225851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab636da5-32d4-4e45-8705-74d8a71cb780 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.261445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.261645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.262100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.262272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.262368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.262528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.262691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.262783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.263353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.263464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:27.263767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.265057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.265389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.275097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 500/1014] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:27.275927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:27.304139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 515/1015] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:27 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:27.385053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:27.395946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e2e07-703b-486e-8d9f-e9b67826d6c7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.432673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.433348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.433628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.433894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.434611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:27.434913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.436316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616623-3bbd-42ca-b7e2-4b4afb1a9f0d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.441885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 501/1016] 10.222.0.23 () {56 vars in 1112 bytes} [Mon Jun 20 06:55:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:27.447652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:27.481351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:27.559482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f05d6-cbba-45b2-9e04-939ffe48116a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:27.593286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:27.593938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:27.594218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:27.594493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:27.595224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:27.596610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d5a3e-43af-4069-a325-abe42a819263 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:27.620144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 516/1017] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:28.451196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574089-f10f-4ef8-8164-46848141b4ce None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:28.451716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574089-f10f-4ef8-8164-46848141b4ce None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:28.451797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61574089-f10f-4ef8-8164-46848141b4ce None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:28.453330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 502/1018] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:28.458709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84996-707b-46e0-b1fc-25cc22e2e28d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:28.459245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84996-707b-46e0-b1fc-25cc22e2e28d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:28.459324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a84996-707b-46e0-b1fc-25cc22e2e28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:28.476648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e0a84996-707b-46e0-b1fc-25cc22e2e28d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:28.477838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e0a84996-707b-46e0-b1fc-25cc22e2e28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:28.549360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 517/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:28 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:28.572069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574089-f10f-4ef8-8164-46848141b4ce None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:28.672449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61574089-f10f-4ef8-8164-46848141b4ce None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:28.715987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:28.716720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:28.717006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:28.717337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:28.718070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:28.719453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ea2f82-0c9c-4e26-9e72-250ef115441d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:28.740135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 503/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:28 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:29.398263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d38da-63b1-444e-9f7d-f344a96612ec None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.398662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d38da-63b1-444e-9f7d-f344a96612ec None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.398840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d38da-63b1-444e-9f7d-f344a96612ec None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.400974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 518/1021] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:29.405923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.406224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.406512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.423578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:29.424804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:29.435827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:29.503724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 504/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:29.528343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d38da-63b1-444e-9f7d-f344a96612ec None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:29.539435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:29.631340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d38da-63b1-444e-9f7d-f344a96612ec None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:29.664638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:29.665357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.665642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.666452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.667627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:29.669123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b44114-f589-4c41-b1a3-cd17c46492d0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:29.675666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 519/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:29.676632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:29.757014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7adec-1475-4071-957f-779b3bb21372 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.757324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7adec-1475-4071-957f-779b3bb21372 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.757581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7adec-1475-4071-957f-779b3bb21372 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.758958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 520/1024] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:29.764694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.765173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.765645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.781468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc2f0a-70ad-476e-b56c-acc6175c6e65 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:29.782483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:29.784086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:29.826253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:29.827043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:29.827330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:29.827608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:29.828368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:29.829843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:29.840024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:29.855245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 505/1025] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:55:29 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:29.866911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 521/1026] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:29 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:29.892197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.001181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e9a7-25c1-4603-8278-2b2a9d243c3b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.038383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.039206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.039494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.039772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.050071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:30.058700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:30.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:30.060619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 506/1027] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:30.067827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.075372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:30.173409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7e049-6f85-429f-b373-4ce62d05034c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.211097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.211768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.220133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.220133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.220133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:30.220133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:30.220133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:30.232411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 522/1028] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:30.232591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.315643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:30.364929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00468ea-ff8c-4e4c-b750-fdad6a60fa17 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.400866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.401662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.401948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.402229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.413562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:30.415030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:30.433466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:30.434423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:30.435363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:30.435363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:30.435363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:30.435363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:30.440776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 507/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:30.440925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.511004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:30.546575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9a35a-8e10-4435-b5aa-34c40b0c6f0d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.581145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.581723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.582014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.582291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.583009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:30.583315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:30.584702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:30.597137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 523/1030] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:30.602469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.712019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8775e-4d6f-43c5-8a13-d2946380fd3d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.722283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:30.748845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.749687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.749976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.750254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.757873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:30.759314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-085ac2ea-d472-4f86-90f2-d42b82155274 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:30.773510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:30.774500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:30.776011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 508/1031] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:30.782263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.789102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a890-b738-449d-9220-55f56987e7c3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.789397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a890-b738-449d-9220-55f56987e7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.789683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315a890-b738-449d-9220-55f56987e7c3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.791736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 509/1032] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:30.796363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.796654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.796937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.813606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:30.814791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:30.887786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68cd83-9aef-4d1e-bfa2-eb77964911a4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:30.896419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 510/1033] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:30 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:30.913143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:30.915437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:30.926404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:30.927237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:30.927525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:30.927806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:30.928682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:30.928994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:30.930433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:30.948314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 524/1034] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:30.953761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.021704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.026036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268b8d4-17cc-4b57-8823-29c1fa903a33 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.061667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb68d5f-40ae-4bb4-80f4-3258190aaf17 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.063588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.064418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.064701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.064984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] PATH_INFO: `/v3/services/image` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.065646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.067016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:31.079159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:31.080237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:31.081112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 511/1035] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:31.086096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.103350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.104082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.104495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.104777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.163157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.178085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.179767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.185793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bca4c-aa3b-4925-8ebe-8e832bee5d22 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.221339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.222002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.222289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.222569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.223279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] RBAC: Adding query filter params (name=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:31.223584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.224981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.230211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 512/1036] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:31.235641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.263714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.291628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 525/1037] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:30 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:31.292333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.341519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933866b1-21a4-4830-9146-e267f8f6b85d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.375915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.376609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.376896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.377174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.377881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] RBAC: Adding query filter params (type=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:31.378191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.379567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.385096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 513/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:31.385327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.390895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.402242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad9ec9-2762-4c15-ac79-8860e0d9ef74 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.440089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.440951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.441239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.441517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.453782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.455361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:31.480315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:31.481509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:31.482745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:31.482745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:31.482745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:31.482745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:31.484199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 526/1039] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:31.489327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.504341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.516038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc0be2-5b11-4815-bf53-fc5a9d2c639e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.550235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.550899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.551188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.551469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.552302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] RBAC: Adding query filter params (interface=public, service_id=c479a16f540647ca910eefb47d566bde, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:31.552621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.553990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.564349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 514/1040] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/endpoints?service_id=c479a16f540647ca910eefb47d566bde&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:31.565329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.594348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394c9c-8f24-4014-9e38-1c2d8e5b47ee None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.630245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.630911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.631200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.631485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.632234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:31.632549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.634032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.641270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.652793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 527/1041] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:31.654978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.674299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca40e5-cefa-443a-b24a-c1d3c53e5ac4 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.716789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.717660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.717949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.718232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.718922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.720325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.727296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 515/1042] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:55:31.765399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96fce5-50ba-4949-b291-3d6938dd9f22 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.805527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.806915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.807205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.807481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.816496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.817966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-540b124f-23e0-4a28-bdb1-c5587d8a6593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:31.834522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:31.835527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:31.841229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 528/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:31.844329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:31.866469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:31.955972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c55142-ac53-437a-9ad7-0806bc091f2b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:31.985616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56bf4-5f49-496b-a502-f228b1f0c338 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.985981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56bf4-5f49-496b-a502-f228b1f0c338 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.986268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b56bf4-5f49-496b-a502-f228b1f0c338 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.988620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 529/1044] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:31.991295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:31.991951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.992328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.992617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.993408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:31.993744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:31.994070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:31.994359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:31.994634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:31.995164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:32.009381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 516/1045] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:32.015166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:32.015338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:32.017684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:32.078951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:32.096451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 530/1046] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:32.097756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:32.126042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:32.129379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9126667-e053-42cc-9477-606c2da1ef57 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:32.167009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:32.167672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:32.167954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:32.168257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:32.169093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] RBAC: Adding query filter params (role.id=32ed5330175c4f55a7f3b7b320dcbafa, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:32.169411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:32.170844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:32.177212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 517/1047] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=32ed5330175c4f55a7f3b7b320dcbafa => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:32.233736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e096f-fa53-4d06-8e14-31f8ad78bd05 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:32.270866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:32.271676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:32.271960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:32.272279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:32.284374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:32.292890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:32.293919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:32.294962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 531/1048] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:32.300248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:32.408705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680f7e2-d23f-4010-9e35-d95e7e623874 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:32.443270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:32.443940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:32.444259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:32.444540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:32.445288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:32.445598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:32.447602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:32.461555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 518/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:32.463352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:32.514151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:32.565993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35bee7-32fd-4c2b-a51b-20b6ec0a8df5 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:32.599495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:32.600395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:32.600686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:32.600967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] PATH_INFO: `/v3/users/586cc408fff345168054a692181c9af2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:32.614512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] RBAC: Authorizing `identity:get_user(user_id=586cc408fff345168054a692181c9af2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:32.615953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:32.620290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 532/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/users/586cc408fff345168054a692181c9af2 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:32.623379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:32.627268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:32.730324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce63ffc-5c3c-4b8b-ba72-045c1344ff70 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:32.765257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:32.765904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:32.766191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:32.766471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:32.835168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:32.836576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:32.840033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:32.946777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 519/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:32.948882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:33.045132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ca5fe-f039-43b6-8e2f-99c6941d8851 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:33.052187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:33.080014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:33.080896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:33.081183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:33.081459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:33.082624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:33.084041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed575c3-40d2-45f7-bd5e-a9754ae2cf90 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:33.085953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 533/1052] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:32 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:33.091710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:33.189796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced319e-279a-4375-bfc1-dbf51dbff146 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:33.192096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:33.225549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:33.226325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:33.226701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:33.227029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:33.227932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=3a22d35bd26f46a2a02e789d58d337c7, user.id=586cc408fff345168054a692181c9af2) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:33.228870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:33.230354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e070d-05d9-428a-b524-ea1346510d26 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:33.236215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 520/1053] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:33 2022] GET /identity/v3/role_assignments?user.id=586cc408fff345168054a692181c9af2&scope.project.id=3a22d35bd26f46a2a02e789d58d337c7&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:33.812321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fc493-5ec0-4142-a747-5a842301a885 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:33.813187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fc493-5ec0-4142-a747-5a842301a885 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:33.813468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fc493-5ec0-4142-a747-5a842301a885 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:33.814865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 534/1054] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:33.820156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:33.820547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:33.820723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:33.838509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:33.839686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:33.907769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 521/1055] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:33 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:33.930649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fc493-5ec0-4142-a747-5a842301a885 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.038880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fc493-5ec0-4142-a747-5a842301a885 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.072467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.073324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.073567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.073846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.074524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.075889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-161ca34a-3731-4190-bbad-a73ea2e0b4d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:34.087134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:34.088773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:34.094554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 535/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Mon Jun 20 06:55:33 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:34.097077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.193035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.197977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e4f51-cbbb-4f6f-95af-fb2aa38b64f1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.200297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da641-f351-49dc-8e55-e82cd1ec1a56 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.200590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da641-f351-49dc-8e55-e82cd1ec1a56 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.200876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44da641-f351-49dc-8e55-e82cd1ec1a56 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.202249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 536/1057] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:34.206951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.207239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.207519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.226431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:34.228270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:34.238228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.238894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.239178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.239457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.240225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] RBAC: Adding query filter params (name=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:34.240495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.241865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.251708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 522/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:34.253262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.266126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.307554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 537/1059] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:34.332049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.362747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbbb31-ffdf-440e-8d8b-c5c13aeaa825 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.400299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.400963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.401249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.401526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.402234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] RBAC: Adding query filter params (type=image) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:34.402539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.403901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 523/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:34.414728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.444558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903cd61-2d65-44b5-bded-7deb123cdbf8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.454009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.484051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.485044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.485328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.485600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.496393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:34.504469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:34.505447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:34.506478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 538/1061] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:34.512138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.537663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264ab3f-712e-4f68-aabc-e899ff4ced25 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.542018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.575138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.575717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.576005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.576321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.577060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.578448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.597601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 524/1062] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:55:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:34.619958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c0839-27c5-445d-966a-d44923fbcac7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.664171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.664854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.665136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.665942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.666716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:34.667023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.668448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.683385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 539/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:34.684628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.779117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.800678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9c5a0-c26a-4317-bd40-f262caecd7d0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:34.835416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:34.836203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:34.836443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:34.836716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:34.847634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:34.849108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:34.866789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:34.867730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:34.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:34.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:34.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:34.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:34.873842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 525/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:34.874460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:34.960097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:34.976450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de31b0-70ba-417f-9472-c39154698e7b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.013519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.014447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.014732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.015010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.015734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:35.016043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.017437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.036154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 540/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:35.037363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.136845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b0929-cee9-49ea-9d7e-e9a19921cf06 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.170940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.171731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.172018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.172323 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.174241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.181553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.183001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-aac3aa60-b9c9-4af8-8824-6338b507c844 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:35.196821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:35.197779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:35.203321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 526/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:35.203923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.253380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a8c3-d3ef-4e24-906f-265e07449992 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.253672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a8c3-d3ef-4e24-906f-265e07449992 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.253951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1a8c3-d3ef-4e24-906f-265e07449992 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.255453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 527/1067] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:35.261390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.261760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.262046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.279393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:35.280621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:35.306986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659c696-0abf-489d-8aa4-278e923498ab None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.342839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.343499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.343785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.344061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.345038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:35.345349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.346895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.352947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 528/1068] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:35.368941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 541/1069] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:35.370119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.376879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.381509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.477708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1861662-965f-4814-aa9c-38dff5c05694 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.481499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d9f6-115a-46fa-9d12-159196fe7068 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.501839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.514117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.514902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.515201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.515457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.516739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.517701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.518152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.518430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.526305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.527768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.529307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:35.536825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:35.537785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:35.538663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 542/1070] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:35.544065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:35.548064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:35.549108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:35.550257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:35.550257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:35.550257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:35.550257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:35.553720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 529/1071] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:35.554524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.610582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.651204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0027e320-2b7c-43e3-b144-1dfa0a8b90a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.654698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb5058-8725-40e0-b3a2-32c895cd12cf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.684996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.685658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.685941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.686217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.686961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:35.687268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.688411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.688669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.689062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.689343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.689615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.690334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:35.690703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.691984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.703273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 543/1072] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:35.709084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.713007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 530/1073] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:35.717994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.759070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.827728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f574db-5802-48e5-a331-7a1724b570e1 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.828854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df54c4c-45cd-4232-bb1c-a0fb8904f08d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:35.856994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:35.863907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.864770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.865093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.865329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.867959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:35.869423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:35.869707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:35.869983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] PATH_INFO: `/v3/users/586cc408fff345168054a692181c9af2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:35.871189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] RBAC: Authorizing `identity:get_user(user_id=586cc408fff345168054a692181c9af2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.872625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:35.872731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.874047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:35.879154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 544/1074] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/users/586cc408fff345168054a692181c9af2 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:35.879786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:35.892790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:35.893728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:35.894637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 531/1075] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:35.899531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:35.982079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023cf7d-6e47-4575-bb41-e7a0942a9593 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:36.000297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:36.007273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58804f4d-2891-4c83-83a8-0a63e7fc22da None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:36.017978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:36.018763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.019242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.019354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.020617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:36.022051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:36.032834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 545/1076] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:36.034827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:36.051937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:36.052624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.052905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.053182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.053964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:36.054274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:36.055703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:36.068555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 532/1077] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:36.076617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:36.111469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:36.153942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14668238-a5f6-4eab-9fc6-9d807fee85a6 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:36.181972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bba08-efbc-4b15-8a49-0c7a6215c365 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:36.189754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:36.190627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.191179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.191179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7/users/586cc408fff345168054a692181c9af2/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.193283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] RBAC: Authorizing `identity:create_grant(project_id=3a22d35bd26f46a2a02e789d58d337c7, user_id=586cc408fff345168054a692181c9af2, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:36.194700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c131aaf-d4fb-4040-9495-2c5973c7b3bc None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:36.208915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 546/1078] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:36 2022] PUT /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7/users/586cc408fff345168054a692181c9af2/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:55:36.217336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:36.218282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.218564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.218852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/32ed5330175c4f55a7f3b7b320dcbafa` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.221194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=e19cf623e22c485fb1bb8f0ada7114b9, role_id=32ed5330175c4f55a7f3b7b320dcbafa)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:36.222669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3843499-b4ba-40f5-a423-76310fc2fe5f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:36.238997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 533/1079] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:36 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/e19cf623e22c485fb1bb8f0ada7114b9/roles/32ed5330175c4f55a7f3b7b320dcbafa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:55:36.696222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb635cb-5d29-459e-8ae1-a2a6c45d6243 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.696514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb635cb-5d29-459e-8ae1-a2a6c45d6243 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.697531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb635cb-5d29-459e-8ae1-a2a6c45d6243 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.699074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 547/1080] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:36.705265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.705265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.705265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:36.722557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:36.723738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:36.735591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:36.796166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 534/1081] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:36.824297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb635cb-5d29-459e-8ae1-a2a6c45d6243 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:36.927168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb635cb-5d29-459e-8ae1-a2a6c45d6243 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:36.951879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:36.965789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:36.966442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:36.966733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:36.967021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:37.033725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:37.035180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:37.141509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 548/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:37.147466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:37.245636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3225d-80e3-4b3a-a35b-f19635ddb54d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:37.280521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:37.281313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:37.281602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:37.281884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:37.296365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:37.298024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:37.305396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-51969d51-2b46-4537-8dbe-f2d2b45037eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:37.316665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:37.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:37.318547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:37.318547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:37.318547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:37.318547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:37.318547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 535/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:37.324293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:37.351786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:37.440744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb2f50-083b-4ebd-b248-1b4aeddebe48 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:37.474728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:37.475380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:37.475666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:37.475945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:37.476699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:37.477010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:37.478926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d184c6-3f4d-45f1-ae20-0bd30517f31d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:37.490350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 549/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:38.259158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bc6db-6826-4d46-8ec2-ce5cd9b68b60 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.259686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bc6db-6826-4d46-8ec2-ce5cd9b68b60 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.259822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bc6db-6826-4d46-8ec2-ce5cd9b68b60 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.261254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 536/1085] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:38.266724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.267104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.267396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.281799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f239-d127-47da-9b96-af51e7586288 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.282089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f239-d127-47da-9b96-af51e7586288 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.282371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058f239-d127-47da-9b96-af51e7586288 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.283692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 537/1086] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:55:38.284606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:38.285852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:38.288592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.288883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.289167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.305875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:38.307025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:38.352753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 550/1087] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:38.373634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 538/1088] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:55:38.375989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.390131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.397128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.474059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d24cf6-b68b-4ea7-b9b6-6ae5691476fe None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.494048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e450d6-a12b-4f1e-9d91-fd08f57fcf6b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.506743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.507542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.507820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.508092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.518591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:38.520176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:38.527403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:38.528618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:38.529648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 551/1089] 10.222.0.23 () {56 vars in 1120 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:38.533419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.534206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.534483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.534663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.534761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.544840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:38.552424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:38.553577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:38.554650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 539/1090] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:38.559262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.607109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.636750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b1013-90c8-4cdd-a0d1-e8e79f44277b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.658034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4409de-e337-45d9-9242-265a5da87c23 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.670724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.671387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.671673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.671958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.672737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:38.673048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:38.674417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:38.682501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 552/1091] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:38.688028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.695516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.697058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.697541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.697824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.698607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:38.698919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:38.700324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:38.713996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 540/1092] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:38.719490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.769931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.785901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cb03f-2c27-43e5-a90e-24f186cfde96 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.817882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5449d58-2435-4a6e-880f-31d94c14d06d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.818591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.819375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.819664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.819947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.822439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.831538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:38.833029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:38.849617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:38.850535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:38.851581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:38.851581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:38.851581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:38.851581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:38.851581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 553/1093] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:38.856325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.856497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.857198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.857484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.857761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] PATH_INFO: `/v3/users/586cc408fff345168054a692181c9af2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.858976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] RBAC: Authorizing `identity:get_user(user_id=586cc408fff345168054a692181c9af2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:38.860436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:38.866644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 541/1094] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/users/586cc408fff345168054a692181c9af2 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:38.867396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:38.954646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaec822-d8cb-478d-95c7-dcad74e4df4d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.965466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470ab5f-c4cb-4044-befa-3c0ed363421c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:38.982265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:38.987005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:38.987654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:38.987935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:38.988237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:38.988948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:38.989249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:38.990598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:38.999979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.000640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.000926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.001201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.002571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 554/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:39.008057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.052893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.068979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.070414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.107259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2978-1e6d-4908-b2e7-d822ac630146 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.140739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.141623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.141909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.142187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.149125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.150611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:55:39.164683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:39.166398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 06:55:39.167734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:39.168499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 542/1096] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:39.171086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 555/1097] 10.222.0.23 () {56 vars in 1093 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:55:39.171412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.171773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.189312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.272792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a13fd7-d528-4b16-80d6-ca38a66b2d5a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.274234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b058b51-bf19-4a96-bd78-87c95bb1cb63 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.281556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.307345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.308159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.308446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.308726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.309608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.309906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.310258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.310538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.310849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.311339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.311668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:39.311975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.313075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 543/1098] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:39.313415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.318918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.332584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 556/1099] 10.222.0.23 () {56 vars in 1133 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:39.338689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.408406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.423722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07ae11-db24-4b58-b928-2f2937539779 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.436473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f976ae-95b8-4b6f-92d4-72c56ec5b195 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.458239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.458890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.459179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.459457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.460320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=3a22d35bd26f46a2a02e789d58d337c7, user.id=586cc408fff345168054a692181c9af2) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:39.460639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.462677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.468800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 544/1100] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/role_assignments?user.id=586cc408fff345168054a692181c9af2&scope.project.id=3a22d35bd26f46a2a02e789d58d337c7&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:39.470723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.471366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.471654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.471935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.504972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.544373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.545845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.647996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 557/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:39.653830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.754064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe2e1c-e74b-432b-bb99-e08ff24f4ac0 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.786890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.787667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.787949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.788254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.798596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.800036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:55:39.817123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:39.818250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:39.819288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:55:39.819288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:55:39.819288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:55:39.819288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:39.819288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 545/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:39.824533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.885745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:39.923293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308a9ee-5a2a-408c-9f25-886559dc524a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:39.956772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:39.957433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:39.957723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:39.958006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:39.958727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:39.959038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:39.960434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:39.971637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 558/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:55:39.977309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:39.980707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:40.078048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913ee32-8527-4ac2-90c3-c98cdab8d61b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:40.110468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:40.111253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:40.111537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:40.111811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:40.119033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:40.120490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:55:40.134365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:40.135448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:55:40.136453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 546/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:40.141600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:40.239683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7c466-b959-4fe8-a169-61ac9631b1ef None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:40.269393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:40.280842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:40.281505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:40.281791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:40.282070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:40.282853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:40.283167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:40.284623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ea9a7-2c34-472d-b6be-595da0ae30ae None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:40.296969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 559/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:40.302688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:40.362631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:40.400794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646935d3-a52a-4841-8b7d-ca3703a4f268 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:40.433421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:40.434062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:40.434345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:40.434623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:40.435443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] RBAC: Adding query filter params (role.id=32ed5330175c4f55a7f3b7b320dcbafa, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=e19cf623e22c485fb1bb8f0ada7114b9) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:40.435755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:40.437218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74361887-7421-4dfc-ad5a-10e0385e8e42 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:40.443065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 547/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:40 2022] GET /identity/v3/role_assignments?user.id=e19cf623e22c485fb1bb8f0ada7114b9&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=32ed5330175c4f55a7f3b7b320dcbafa => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:41.517863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7af2a-f959-4517-8938-4d3fadab8ae0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:41.518383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7af2a-f959-4517-8938-4d3fadab8ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:41.518505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7af2a-f959-4517-8938-4d3fadab8ae0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:41.519804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 560/1107] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:41.524983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:41.525379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:41.525551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:41.542354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:41.543523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:41.608876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 548/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:41.635833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7af2a-f959-4517-8938-4d3fadab8ae0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:41.733819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7af2a-f959-4517-8938-4d3fadab8ae0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:41.765809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:41.766582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:41.766863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:41.767138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:41.768373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:41.769751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b241ad4d-2c18-4352-b093-c18e1637a1a4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:41.771363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 561/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:41.778368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:41.833906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:41.876066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240d3a6-1a97-4ef1-bc1f-01a4d52bffee None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:41.908864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:41.909517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:41.909803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:41.910084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:41.910793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:41.912247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d933f-ded0-4706-b34b-4002bdbf0ed5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:41.931303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 549/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:55:41 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:43.975700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42bf5-f899-4e91-af84-9869af3ce92c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:43.976250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42bf5-f899-4e91-af84-9869af3ce92c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:43.976250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a42bf5-f899-4e91-af84-9869af3ce92c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:43.977588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 562/1111] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:43.982760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:43.983173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:43.983321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.000050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:44.001259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:44.065931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 550/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:44.092664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a42bf5-f899-4e91-af84-9869af3ce92c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:44.146316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:44.192086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a42bf5-f899-4e91-af84-9869af3ce92c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:44.224465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:44.225297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:44.225580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:44.225900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] PATH_INFO: `/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.227336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] RBAC: Authorizing `identity:get_role(role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:44.228758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:44.230107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 563/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:44 2022] GET /identity/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:44.235638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:44.277830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:44.332891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b836757-30dc-4694-b83b-99ced8cd9ae1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:44.365640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:44.366492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:44.366779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:44.367060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] PATH_INFO: `/v3/users/02b519f5c64944e2b074e3cfb79bc122` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.380255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] RBAC: Authorizing `identity:get_user(user_id=02b519f5c64944e2b074e3cfb79bc122)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:44.381716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:44.383340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 551/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:44 2022] GET /identity/v3/users/02b519f5c64944e2b074e3cfb79bc122 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:44.389217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:44.484344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d8252-b1cc-4c5f-b1ee-b847be2b0cfd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:44.503771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:44.518377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:44.519009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:44.519296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:44.519576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.583530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:44.584990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:44.677321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 564/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:44.683806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:44.778584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:44.781543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcefea8-6d3e-401e-9c9b-3d1a80d5461a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:44.814719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:44.815556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:44.815835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:44.816130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.817297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:44.818714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e71b5c-df89-4b7a-8264-5eb1970d069b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:44.820407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 552/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:44 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:44.826436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:44.847315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:44.922192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8784ee-2426-4738-b1d9-77eef070bea7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:44.954140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:44.954776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:44.955060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:44.955335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:44.956177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=3a22d35bd26f46a2a02e789d58d337c7, user.id=02b519f5c64944e2b074e3cfb79bc122) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:44.956493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:44.957906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15af2a-fe8e-4d7c-932d-6545b27a672d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:44.963717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 565/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:44 2022] GET /identity/v3/role_assignments?user.id=02b519f5c64944e2b074e3cfb79bc122&scope.project.id=3a22d35bd26f46a2a02e789d58d337c7&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:46.961085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014994-e357-42bf-a833-1ac399198498 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:46.961602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014994-e357-42bf-a833-1ac399198498 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:46.961602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee014994-e357-42bf-a833-1ac399198498 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:46.963539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 553/1118] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:46.968804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:46.969200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:46.969387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:46.986206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:46.987374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:47.051195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 566/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:47.073123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee014994-e357-42bf-a833-1ac399198498 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:47.179503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee014994-e357-42bf-a833-1ac399198498 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:47.211490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:47.212302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:47.213105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:47.213382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] PATH_INFO: `/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:47.214847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] RBAC: Authorizing `identity:get_role(role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:47.216246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:47.217616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 554/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:47 2022] GET /identity/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:47.223429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:47.238479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:47.318589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b4765-ecbd-4df0-907d-3babaf1221e8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:47.353767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:47.354623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:47.354903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:47.355177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] PATH_INFO: `/v3/users/02b519f5c64944e2b074e3cfb79bc122` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:47.356364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] RBAC: Authorizing `identity:get_user(user_id=02b519f5c64944e2b074e3cfb79bc122)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:47.357780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:47.359380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 567/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:47 2022] GET /identity/v3/users/02b519f5c64944e2b074e3cfb79bc122 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:47.365115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:47.399752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:47.468558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e59d8e-e13e-489d-9386-1f512fd56f76 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:47.500665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:47.501506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:47.501786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:47.502062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:47.503239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:47.504705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ca8f8-cfe0-4bac-922e-cfaae86cfb82 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:47.506472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 555/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:47 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:47.512904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:47.591071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:47.608447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666b3a-39fe-4185-b444-ae8afd93aeab None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:47.640907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:47.641772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:47.642055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:47.642330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7/users/02b519f5c64944e2b074e3cfb79bc122/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:47.644518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a22d35bd26f46a2a02e789d58d337c7, user_id=02b519f5c64944e2b074e3cfb79bc122, role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:47.645935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe69eb-27ae-419f-9ee4-2d1f7837e248 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:47.658251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 568/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:55:47 2022] PUT /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7/users/02b519f5c64944e2b074e3cfb79bc122/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:55:49.635732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cfcb-aba5-4724-9b27-ed4da0913eba None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:49.636307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cfcb-aba5-4724-9b27-ed4da0913eba None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:49.636307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669cfcb-aba5-4724-9b27-ed4da0913eba None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:49.637700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 556/1124] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:49.642966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:49.643355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:49.643518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:49.659781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:49.660993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:49.725074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 569/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:49.749403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cfcb-aba5-4724-9b27-ed4da0913eba None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:49.849892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669cfcb-aba5-4724-9b27-ed4da0913eba None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:49.852161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:49.883514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:49.884428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:49.884721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:49.885002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] PATH_INFO: `/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:49.886529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] RBAC: Authorizing `identity:get_role(role_id=ffbdbe27a1fb46dba1cd4201c4f9a2bd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:49.887944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:49.889621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 557/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:49 2022] GET /identity/v3/roles/ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:49.895336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:49.961638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:49.991388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f247a-1ec3-4d5e-b1e4-93aba1a636f8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:50.023491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:50.024376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:50.024664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:50.025455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] PATH_INFO: `/v3/users/02b519f5c64944e2b074e3cfb79bc122` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:50.026638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] RBAC: Authorizing `identity:get_user(user_id=02b519f5c64944e2b074e3cfb79bc122)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:50.028066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:50.029716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 570/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:49 2022] GET /identity/v3/users/02b519f5c64944e2b074e3cfb79bc122 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:50.035293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:50.136378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787ef7a-0886-4cc7-9e5c-0af321a86733 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:50.169956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:50.170527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:50.170812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:50.171086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:50.209451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:50.235801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:50.237270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:50.331157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 558/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:55:50.336998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:50.432701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f949e-5c3c-4c1e-ad4b-f1d431fe5d62 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:50.465691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:50.466523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:50.466806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:50.467082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] PATH_INFO: `/v3/projects/3a22d35bd26f46a2a02e789d58d337c7` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:50.468281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] RBAC: Authorizing `identity:get_project(project_id=3a22d35bd26f46a2a02e789d58d337c7)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:50.469705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79602f67-a9b3-49c4-8cfc-68c2de6caa4f None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:50.471370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 571/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:50 2022] GET /identity/v3/projects/3a22d35bd26f46a2a02e789d58d337c7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:50.477276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:50.544101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:50.572943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019ba9c-d98c-49de-83f4-94f411dbd09e None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:50.605520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:50.606149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:50.606433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:50.606710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:50.607532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] RBAC: Adding query filter params (role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd, scope.project.id=3a22d35bd26f46a2a02e789d58d337c7, user.id=02b519f5c64944e2b074e3cfb79bc122) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:50.607842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:50.609297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd5688-d6ce-4e27-8e52-5bdf5f8775cc None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:50.614961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 559/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:50 2022] GET /identity/v3/role_assignments?user.id=02b519f5c64944e2b074e3cfb79bc122&scope.project.id=3a22d35bd26f46a2a02e789d58d337c7&role.id=ffbdbe27a1fb46dba1cd4201c4f9a2bd => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:52.628864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292a113-cb31-4f37-bc8d-613274e89ea0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:52.629417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292a113-cb31-4f37-bc8d-613274e89ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:52.629417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292a113-cb31-4f37-bc8d-613274e89ea0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:52.630768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 572/1131] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:52.635800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:52.636240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:52.636390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:52.652870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:52.654025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:52.725784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 560/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:52.749605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292a113-cb31-4f37-bc8d-613274e89ea0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:52.760302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:52.847060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292a113-cb31-4f37-bc8d-613274e89ea0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:52.879633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:52.880370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:52.880650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:52.880922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:52.882072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:52.883490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4829e2-8cab-418d-8010-326efd761c5a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:52.885129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 573/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:52.891159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:52.953108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:52.988197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13e9f6-0518-405b-aac6-5da58479b0a5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:53.021176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:53.021820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:53.022106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:53.022386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:53.023094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:53.025167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db0bd9-7110-4d38-aeba-8125c5b48ed4 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:53.051605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 561/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:55:52 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:55:55.071890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ea23-33e9-48a4-bbb4-f96e51a77b4b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:55.072464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ea23-33e9-48a4-bbb4-f96e51a77b4b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:55.072464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0ea23-33e9-48a4-bbb4-f96e51a77b4b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:55.073840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 574/1135] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:55.079043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:55.079456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:55.079612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:55.096550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:55.097713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:55.162601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 562/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:55.187202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ea23-33e9-48a4-bbb4-f96e51a77b4b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:55.261445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:55.284673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de0ea23-33e9-48a4-bbb4-f96e51a77b4b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:55.317317 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:55.318075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:55.318351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:55.318624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:55.319761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:55.321206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042092c7-8439-47c1-a7cd-e41786e9e587 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:55.323425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 575/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:55:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:55.329009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:55.375319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:55.424046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beba534-de6a-4d89-bef7-48e687d8e335 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:55.457560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:55.458213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:55.458496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:55.458772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:55.459484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:55.460931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efb022b-670a-419d-b68e-dabf03fef547 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:55.479200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 563/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:55:55 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:55:57.512417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977edf6-9cab-49af-80d8-501b259176f0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:57.512973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977edf6-9cab-49af-80d8-501b259176f0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:57.512973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977edf6-9cab-49af-80d8-501b259176f0 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:57.514592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 576/1139] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:55:57.519753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:57.520191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:57.520346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:57.536739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:55:57.537919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:55:57.602477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 564/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:55:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:55:57.625395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977edf6-9cab-49af-80d8-501b259176f0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:57.724672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977edf6-9cab-49af-80d8-501b259176f0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:57.757412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:57.758190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:57.758474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:57.758787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:57.768943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:55:57.776970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:55:57.778155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:55:57.779296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 577/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:55:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:55:57.784213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:57.816515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:57.880129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974214c-1d74-4d9d-ad92-dedfc25281ba None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:57.914351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:57.915023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:57.915305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:57.915583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:57.916347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:57.916654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:57.918021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:57.925746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 565/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:55:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:57.931496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:57.935208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:58.028277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b86421-e46a-4653-9e22-dfa07086a71d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:58.059497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:58.060394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:58.060674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:58.060945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] PATH_INFO: `/v3/users/07ede7bc2c6440e385819277ce8484e9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:58.073940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] RBAC: Authorizing `identity:get_user(user_id=07ede7bc2c6440e385819277ce8484e9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:58.075357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:58.076997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 578/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:55:57 2022] GET /identity/v3/users/07ede7bc2c6440e385819277ce8484e9 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:58.082550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:58.173759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:58.179729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0927cb47-1750-4bc5-a873-1d09eb2b9ea9 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:58.212215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:58.212846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:58.213126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:58.213398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:58.277670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:58.279153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:58.366155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:58.373504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 566/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:55:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:55:58.379338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:58.438344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:55:58.476223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3731572-e2e1-4c8a-87bc-da66d115f492 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:58.509406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:58.510258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:58.510545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:58.510824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] PATH_INFO: `/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:58.511996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] RBAC: Authorizing `identity:get_project(project_id=45b0b74dce10432ab02d3ede06f9a1d5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:58.513707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a40ff-d465-442f-9051-0cad56f18f28 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:58.515398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 579/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:55:58 2022] GET /identity/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:55:58.521433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:55:58.615850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a075634-e9be-4ef1-9c31-c03015e5e3cf None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:55:58.648307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:55:58.648964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:55:58.649248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:55:58.649523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:55:58.650334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=45b0b74dce10432ab02d3ede06f9a1d5, user.id=07ede7bc2c6440e385819277ce8484e9) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:55:58.650645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:55:58.652274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ade58-856e-4f5b-9cec-6997283e7ce3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:55:58.657585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 567/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:55:58 2022] GET /identity/v3/role_assignments?user.id=07ede7bc2c6440e385819277ce8484e9&scope.project.id=45b0b74dce10432ab02d3ede06f9a1d5&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:00.641314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed3b66-58a3-47e3-a347-6452d85fb1b2 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:00.641847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed3b66-58a3-47e3-a347-6452d85fb1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:00.641847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed3b66-58a3-47e3-a347-6452d85fb1b2 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:00.643237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 580/1147] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:00.648478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:00.648899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:00.649030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:00.665349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:00.666506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:00.720333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:00.731355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 568/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:56:00.754535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed3b66-58a3-47e3-a347-6452d85fb1b2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:00.851129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed3b66-58a3-47e3-a347-6452d85fb1b2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:00.878843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:00.884990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:00.885762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:00.886044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:00.886317 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:00.896393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:56:00.903914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:00.905084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:56:00.906148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 581/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:00.911162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:01.006431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258407d-d0d0-4208-a7df-29f3acd21a42 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:01.038683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:01.039329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:01.039608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:01.040426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:01.041157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:01.041459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:01.042813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:01.050580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 569/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:56:01.056858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:01.156788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e18dbb-5d78-4caa-ac0e-be903f54ad61 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:01.189194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:01.190041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:01.190323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:01.190597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] PATH_INFO: `/v3/users/07ede7bc2c6440e385819277ce8484e9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:01.191754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] RBAC: Authorizing `identity:get_user(user_id=07ede7bc2c6440e385819277ce8484e9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:01.193200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:01.194814 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 582/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:56:01 2022] GET /identity/v3/users/07ede7bc2c6440e385819277ce8484e9 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:01.200572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:01.252151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:01.298692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094002e-34f9-4942-82b8-ced93781e7ce None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:01.330026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:01.330863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:01.331143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:01.331418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] PATH_INFO: `/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:01.332594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] RBAC: Authorizing `identity:get_project(project_id=45b0b74dce10432ab02d3ede06f9a1d5)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:01.334009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99952-f711-41a8-a2c0-7c4c210e7327 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:01.335660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 570/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:56:01 2022] GET /identity/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:01.341851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:01.398480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:01.439335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de78da1-f94a-45a8-a2b3-f9e4e16e3245 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:01.471799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:01.472697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:01.472982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:01.473257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] PATH_INFO: `/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5/users/07ede7bc2c6440e385819277ce8484e9/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:01.475203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] RBAC: Authorizing `identity:create_grant(project_id=45b0b74dce10432ab02d3ede06f9a1d5, user_id=07ede7bc2c6440e385819277ce8484e9, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:01.476656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f8e9a-b8d2-48a9-8fe8-32ac5f72245e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:01.489928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 583/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:56:01 2022] PUT /identity/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5/users/07ede7bc2c6440e385819277ce8484e9/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:56:03.454530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d087a-ee8f-4710-a93b-81ee35de0488 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:03.455083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d087a-ee8f-4710-a93b-81ee35de0488 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:03.455083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d087a-ee8f-4710-a93b-81ee35de0488 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:03.456507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 571/1154] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:03.462808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:03.463207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:03.463371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:03.479910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:03.481159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:03.544622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 584/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:56:03.568796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d087a-ee8f-4710-a93b-81ee35de0488 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:03.667516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d087a-ee8f-4710-a93b-81ee35de0488 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:03.687119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:03.701678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:03.702472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:03.702757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:03.703039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:03.713149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:56:03.720745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:03.721841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:56:03.722901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 572/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:03.727999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:03.775182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:03.824502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c04c8a-bf1f-4ea1-ac5a-e29691abacbd None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:03.856996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:03.857641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:03.857922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:03.858196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:03.858931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:03.859239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:03.860640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:03.869005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 585/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:03.874651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:03.972248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71357cd2-9d41-4050-b69b-856269682c8b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:04.005458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:04.006312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:04.006598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:04.006874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] PATH_INFO: `/v3/users/07ede7bc2c6440e385819277ce8484e9` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:04.008055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] RBAC: Authorizing `identity:get_user(user_id=07ede7bc2c6440e385819277ce8484e9)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:04.009511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:04.011127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 573/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:56:03 2022] GET /identity/v3/users/07ede7bc2c6440e385819277ce8484e9 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:04.017156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:04.112174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd96788-be0d-4f35-8f72-5da375e69900 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:04.122479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:04.145406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:04.146035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:04.146323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:04.146603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:04.217637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:04.219078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:04.313082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 586/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 06:56:04.318851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:04.366968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:04.416774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b972b19f-17e4-46f5-8ae3-7ba5e5fbf92a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:04.448472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:04.449324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:04.449621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:04.449890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] PATH_INFO: `/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:04.451098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] RBAC: Authorizing `identity:get_project(project_id=45b0b74dce10432ab02d3ede06f9a1d5)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:04.452561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6189ca0-d5b8-4cf2-af77-5856d123420d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:04.454243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 574/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:56:04 2022] GET /identity/v3/projects/45b0b74dce10432ab02d3ede06f9a1d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:04.460178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:04.471781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:04.555219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce61c2-0ccd-4b79-a1a5-7def91944cc0 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:04.588588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:04.589160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:04.589441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:04.589713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:04.590518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=45b0b74dce10432ab02d3ede06f9a1d5, user.id=07ede7bc2c6440e385819277ce8484e9) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:04.590823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:04.592251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96420311-bb42-4038-946e-7ebca0ad2056 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:04.597821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 587/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:56:04 2022] GET /identity/v3/role_assignments?user.id=07ede7bc2c6440e385819277ce8484e9&scope.project.id=45b0b74dce10432ab02d3ede06f9a1d5&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:06.619100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7c124-3e3b-4b8d-a64b-dfd2c7aab171 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:06.619628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7c124-3e3b-4b8d-a64b-dfd2c7aab171 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:06.619628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7c124-3e3b-4b8d-a64b-dfd2c7aab171 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:06.621051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 575/1162] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:06.626187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:06.626601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:06.626738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:06.643883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:06.645083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:06.707763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 588/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:06.729421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7c124-3e3b-4b8d-a64b-dfd2c7aab171 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:06.828293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7c124-3e3b-4b8d-a64b-dfd2c7aab171 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:06.843818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:06.861497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:06.862131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:06.862415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:06.862691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:06.927043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:06.928459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:07.022799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 576/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:56:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:56:07.028851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:07.031847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:07.122788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91757b9-2772-40a1-b337-bc2c969bb7f5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:07.154523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:07.155297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:07.155577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:07.155848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:07.162223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:07.163656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ff11db09-09ed-4ae1-a540-fb196b6bc7c8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:07.177926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:07.179212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:56:07.180089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 589/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:56:07 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:07.185362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:07.202180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:07.280999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b46a5-773e-478a-ad80-0bc7bb1f3e94 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:07.313707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:07.314342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:07.314623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:07.314897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:07.315611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:07.315912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:07.317293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03837de9-dd19-4b77-8b13-fca99c4e1097 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:07.324230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 577/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Mon Jun 20 06:56:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:09.335594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2e38-65c4-4d16-96a4-9242a7982b8f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:09.336138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2e38-65c4-4d16-96a4-9242a7982b8f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:09.336233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2e38-65c4-4d16-96a4-9242a7982b8f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:09.337565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 590/1167] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:09.342849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6cb5-4387-4760-a3ac-06026dce8fbc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:09.343252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6cb5-4387-4760-a3ac-06026dce8fbc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:09.343419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb6cb5-4387-4760-a3ac-06026dce8fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:09.360360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8efb6cb5-4387-4760-a3ac-06026dce8fbc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:09.361545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8efb6cb5-4387-4760-a3ac-06026dce8fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:09.426980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 578/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:09.449744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2e38-65c4-4d16-96a4-9242a7982b8f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:09.511754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:09.551053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2e38-65c4-4d16-96a4-9242a7982b8f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:09.583269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:09.583913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:09.584224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:09.584504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:09.585239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:09.587192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5720ef-ddb7-428c-aaf8-ee7f0c92dcd1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:09.600697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 591/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Mon Jun 20 06:56:09 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:56:11.639900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f932-c9d2-41e2-827b-160d1332e887 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:11.640468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f932-c9d2-41e2-827b-160d1332e887 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:11.640468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6f932-c9d2-41e2-827b-160d1332e887 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:11.641871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 579/1170] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:11.647047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:11.647479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:11.647583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:11.663833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:11.665591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:11.731041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 592/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:11.754223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f932-c9d2-41e2-827b-160d1332e887 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:11.766637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:11.858529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6f932-c9d2-41e2-827b-160d1332e887 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:11.890980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:11.891756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:11.892034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:11.892336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] PATH_INFO: `/v3/domains/a0338713849340bf8ce7aee30b18fd81` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:11.902929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0338713849340bf8ce7aee30b18fd81)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:11.904410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4023239-b4ce-463e-85e2-640fee6e8e6e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:11.906064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 580/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Mon Jun 20 06:56:11 2022] GET /identity/v3/domains/a0338713849340bf8ce7aee30b18fd81 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:11.911901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:11.964646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:12.008540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb8259-8a5c-47c8-b94a-e03c92b914d8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:12.041921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:12.042562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:12.042849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:12.043128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:12.043840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:12.045286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0891-1edd-46d2-96ee-b15b9acc9691 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:12.069110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 593/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:56:11 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:56:14.078832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b848205-a5fa-4143-8dc4-ad0fc58b4e0b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:14.079347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b848205-a5fa-4143-8dc4-ad0fc58b4e0b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:14.079464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b848205-a5fa-4143-8dc4-ad0fc58b4e0b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:14.080795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 581/1174] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:14.085754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:14.086693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:14.086886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:14.104402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:14.105581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:14.176295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 594/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:14.200550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b848205-a5fa-4143-8dc4-ad0fc58b4e0b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:14.267768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:14.299023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b848205-a5fa-4143-8dc4-ad0fc58b4e0b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:14.331220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:14.332083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:14.332390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:14.332661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] PATH_INFO: `/v3/domains/a0338713849340bf8ce7aee30b18fd81` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:14.333819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=a0338713849340bf8ce7aee30b18fd81)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:14.335233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d9cac6-73fb-4b9a-91b6-c16debc1a4ab None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:14.337403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 582/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Mon Jun 20 06:56:14 2022] GET /identity/v3/domains/a0338713849340bf8ce7aee30b18fd81 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:14.343010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:14.374954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:14.437682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3f2b8-9d02-43f4-9283-e04af5bf071d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:14.470401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:14.471051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:14.471335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:14.471610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:14.472338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:14.473754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0d54-c06f-434a-b3b1-80ce95a117f6 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:14.492853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 595/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:56:14 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:56:16.528457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdaa0-7df0-4154-9b55-4106e25907c9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:16.529040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdaa0-7df0-4154-9b55-4106e25907c9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:16.529040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135fdaa0-7df0-4154-9b55-4106e25907c9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:16.530383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 583/1178] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:16.535570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:16.535986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:16.536169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:16.552924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:16.554103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:16.618854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 596/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:16.642059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fdaa0-7df0-4154-9b55-4106e25907c9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:16.745537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135fdaa0-7df0-4154-9b55-4106e25907c9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:16.777676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:16.778451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:16.778783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:16.779011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:16.785599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:16.790223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:56:16.797888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:16.799115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:56:16.800369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 584/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:56:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:16.805590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:16.828028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:16.900404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c099e-4e03-4ec4-9e39-4242b8e87ea0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:16.932921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:16.933567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:16.933848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:16.934122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:16.934858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:16.935163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:16.936555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:16.944191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 597/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:56:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:16.949967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:17.047394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8bf4fd-e706-458c-8879-f6f50cbc3842 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:17.078707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:17.079563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:17.079846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:17.080131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] PATH_INFO: `/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:17.093152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] RBAC: Authorizing `identity:get_user(user_id=285fc9c6cc4f48ffb2a979f6326a12fb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:17.094575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:17.096195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 585/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:56:16 2022] GET /identity/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:17.102283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:17.182127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:17.197659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527320a-2bc9-4270-8e29-d1d123445f9b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:17.229880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:17.230509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:17.230790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:17.231066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:17.294115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:17.295536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:17.372597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:17.389104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 598/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:56:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:56:17.395008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:17.450223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:17.494249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb15b-b9fb-4c81-af9d-ba39e7f553fe None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:17.535815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:17.536698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:17.536984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:17.537260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] PATH_INFO: `/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:17.538437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7f129b8d4479a8e056b4bbbc7c2fb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:17.539903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad4a68-969e-4f7c-abc5-528930b8058e None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:17.541638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 586/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:56:17 2022] GET /identity/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:17.547724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:17.641869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5826b0-8fd8-4931-a022-754fea276a4b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:17.673609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:17.674235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:17.674514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:17.674786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:17.675590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=e9d7f129b8d4479a8e056b4bbbc7c2fb, user.id=285fc9c6cc4f48ffb2a979f6326a12fb) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:17.675894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:17.677337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173358a-8bb2-4892-b237-1d4c50bb5efb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:17.683065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 599/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:56:17 2022] GET /identity/v3/role_assignments?user.id=285fc9c6cc4f48ffb2a979f6326a12fb&scope.project.id=e9d7f129b8d4479a8e056b4bbbc7c2fb&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:19.705862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39290f3-db73-4b15-be0f-a69c56f9798b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:19.706400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39290f3-db73-4b15-be0f-a69c56f9798b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:19.706960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39290f3-db73-4b15-be0f-a69c56f9798b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:19.707873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 587/1186] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:19.713190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:19.713613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:19.713751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:19.730392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:19.731564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:19.772948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:19.795401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 600/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:19.818403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39290f3-db73-4b15-be0f-a69c56f9798b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:19.919175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39290f3-db73-4b15-be0f-a69c56f9798b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:19.941737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:19.952245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:19.953013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:19.953294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:19.953568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:19.963846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:56:19.971195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:19.972491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:56:19.973752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 588/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:56:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:19.978899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:20.072880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cba16f8-5e5c-4fe6-87e1-fa3dd0ee0727 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:20.107104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:20.107761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:20.108048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:20.108354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:20.109096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:20.109403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:20.110769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:20.119079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 601/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:56:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:20.124844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:20.225788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603c2f8-b332-488e-b3aa-183588a462aa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:20.262183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:20.263300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:20.263581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:20.263852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] PATH_INFO: `/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:20.265047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] RBAC: Authorizing `identity:get_user(user_id=285fc9c6cc4f48ffb2a979f6326a12fb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:20.266474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:20.268377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 589/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:56:20 2022] GET /identity/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:20.274775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:20.315446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:20.371999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018578ed-ea48-44f1-b6b3-67cb3d52b56e None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:20.404806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:20.405645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:20.405925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:20.406198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] PATH_INFO: `/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:20.407341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7f129b8d4479a8e056b4bbbc7c2fb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:20.408784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8304e0-7ced-4412-936c-1d82f8ed1218 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:20.410446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 602/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:56:20 2022] GET /identity/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:56:20.416249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:20.468130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:20.518481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df6d12-af0d-4cf9-a904-54d3ad7141ea None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:20.550737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:20.551611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:20.551894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:20.552201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] PATH_INFO: `/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb/users/285fc9c6cc4f48ffb2a979f6326a12fb/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:20.554149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d7f129b8d4479a8e056b4bbbc7c2fb, user_id=285fc9c6cc4f48ffb2a979f6326a12fb, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:20.555604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10c1e10-6cd9-4576-a8ae-7f57346b5b21 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:20.567668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 590/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:56:20 2022] PUT /identity/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb/users/285fc9c6cc4f48ffb2a979f6326a12fb/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:56:22.559905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9d95-8204-45d2-a8ea-c206e55058b4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:22.560483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9d95-8204-45d2-a8ea-c206e55058b4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:22.560483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee9d95-8204-45d2-a8ea-c206e55058b4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:22.561837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 603/1193] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:56:22.566995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:22.567428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:22.567570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:22.584024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:56:22.585228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:56:22.649932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 591/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:56:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:56:22.672875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee9d95-8204-45d2-a8ea-c206e55058b4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:22.769897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee9d95-8204-45d2-a8ea-c206e55058b4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:22.778587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:22.802725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:22.803489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:22.803767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:22.804034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:22.814285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:56:22.821727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:56:22.822846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:56:22.823973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 604/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:56:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:56:22.828794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:22.871552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:22.926414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca4cfb-3042-449d-9e62-278e74c6922d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:22.963496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:22.964174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:22.964465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:22.964747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:22.965491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:22.965801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:22.967165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:22.974949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 592/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:56:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:22.981328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:23.076874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e0862-0359-4770-b75f-f3904389d31c None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:23.109325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:23.110166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:23.110449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:23.110725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] PATH_INFO: `/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:23.111886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] RBAC: Authorizing `identity:get_user(user_id=285fc9c6cc4f48ffb2a979f6326a12fb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:23.113341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:23.114931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 605/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Mon Jun 20 06:56:22 2022] GET /identity/v3/users/285fc9c6cc4f48ffb2a979f6326a12fb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:23.120436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:23.218116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5da48-1a3c-4f9c-80d8-4c978af26926 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:23.220512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:23.250942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:23.251584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:23.251874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:23.252182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:23.317459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:23.318905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:23.414204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 593/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:56:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:56:23.419960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:23.457570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:23.520722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319779-805c-4132-bee2-5e7fd4190cbe None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:23.553005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:23.553853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:23.554137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:23.554452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] PATH_INFO: `/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:23.555615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d7f129b8d4479a8e056b4bbbc7c2fb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:23.557071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a29045-238e-4ddf-b042-592f5479dc13 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:23.558757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 606/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Mon Jun 20 06:56:23 2022] GET /identity/v3/projects/e9d7f129b8d4479a8e056b4bbbc7c2fb => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:23.564822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:56:23.567855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:23.660520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ed12f-b93a-49bb-921c-9c10264e58c2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:56:23.692912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:56:23.693490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:56:23.693768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:56:23.694053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:56:23.694918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=e9d7f129b8d4479a8e056b4bbbc7c2fb, user.id=285fc9c6cc4f48ffb2a979f6326a12fb) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:56:23.695730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:56:23.697175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7151f034-0489-4e20-a642-8ab476ca4b68 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:56:23.703025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 594/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:56:23 2022] GET /identity/v3/role_assignments?user.id=285fc9c6cc4f48ffb2a979f6326a12fb&scope.project.id=e9d7f129b8d4479a8e056b4bbbc7c2fb&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:56:33.548923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=48 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:56:33.700394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=66 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:18.366131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80457d0-c0a5-420f-94d0-8d42b4db4ba6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:18.366131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80457d0-c0a5-420f-94d0-8d42b4db4ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:18.366131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80457d0-c0a5-420f-94d0-8d42b4db4ba6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:18.366131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 607/1201] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:18.371714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:18.372362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:18.372362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:18.392374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:18.394922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:18.492074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 595/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:18.516997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80457d0-c0a5-420f-94d0-8d42b4db4ba6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:18.648415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80457d0-c0a5-420f-94d0-8d42b4db4ba6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:18.690730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:18.691907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:18.692886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:18.693143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:18.706672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:18.708529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:18.729653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:18.731129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:18.733297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:18.733297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:18.733297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:18.733297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:18.733297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 608/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:18.737210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:18.882448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737583a7-63ae-4b14-ac60-c3f9ceb2debb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:18.926114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:18.935737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:18.936610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:18.936831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:18.937184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:18.938698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:18.939028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:18.940711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22616dad-59fc-4389-a7c1-bf5947ff6724 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:18.959876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 596/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:18.971008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:19.044779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:19.118732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf77f-56ee-4a29-b2f4-2c80641aacd1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:19.158536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:19.159369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:19.159647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:19.159920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:19.161389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:19.162833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072790-46b4-4ba2-9560-0a54b2007167 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:19.186726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 609/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Mon Jun 20 06:57:18 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:57:21.480147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90301f-f6e6-48b4-8fc9-257e12a2a562 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:21.480813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90301f-f6e6-48b4-8fc9-257e12a2a562 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:21.480813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90301f-f6e6-48b4-8fc9-257e12a2a562 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:21.482246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 597/1206] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:21.487784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:21.488285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:21.488392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:21.507637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:21.508984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:21.590125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 610/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:57:21.617524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90301f-f6e6-48b4-8fc9-257e12a2a562 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:21.770188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90301f-f6e6-48b4-8fc9-257e12a2a562 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:21.812916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:21.814236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:21.814825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:21.815437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:21.828885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:21.837809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:21.839172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:21.840182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 598/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:57:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:21.845587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:21.895071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:21.968262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb101643-c8f4-40e3-aa5f-2ba8611b991e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.013882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.014561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.014887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.015119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:22.015967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:22.016327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:22.017911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:22.032372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 611/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:57:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:22.038942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:22.103108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:22.158898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d288e4-83a3-442e-84fe-669bea8cde4f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.199001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.200045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.200459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.200774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:22.212229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:22.213691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:22.230935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:22.231895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:22.232867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:22.232867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:22.232867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:22.232867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:22.237979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:22.238417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 599/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:57:22.360755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:22.363504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a4ab7-77bb-4a8b-b888-bca53d6247f8 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.405128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.405795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.406120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.406400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:22.407171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:22.407454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:22.408912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:22.425995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 612/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:22.428477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:22.547092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f84c4b-85e4-47e3-989e-f116379457d2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.549491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:22.592828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.593979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.594365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.594569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:22.603711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:22.605554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:22.623573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:22.624999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:22.626014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 600/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:57:22.633040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:22.757081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7858f9-7ef9-4c83-8fa6-8a65af4651c9 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.798657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.799453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.799798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.800089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:22.801008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:22.802088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:22.803757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:22.829853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 613/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:57:22.832012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:22.956356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21731f7d-6614-40b6-b08f-d1d33be99e45 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:22.996041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:22.996727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:22.997102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:22.997414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:23.031206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:23.075624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:23.078701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:23.201785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 601/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:57:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:57:23.208537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:23.234299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:23.334665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa99a6-887b-437a-99dd-d63d9d74c4d1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:23.370548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:23.372156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:23.372156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:23.372156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:23.384385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:23.386162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:23.408015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:23.409185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:23.410182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:23.410182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:23.410182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:23.410182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:23.411257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 614/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:23.417310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:23.495394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:23.537703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e42c78-5ca8-4c06-beda-8dfc7a931ab3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:23.573562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:23.574345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:23.574564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:23.574916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:23.575637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:23.575978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:23.577384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:23.594039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 602/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:23.595477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:23.673920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:23.713944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b947ae1-9e19-47aa-a29f-832937356f13 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:23.762348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:23.763196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:23.763475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:23.763786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:23.773108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:23.774914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:23.789885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:23.791027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:23.792169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:23.794032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 615/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:23.799860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:23.919440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5492-654a-460c-9858-bed2b320e21c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:23.944166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:23.958734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:23.959398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:23.959693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:23.959963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:23.960803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:23.961116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:23.962552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd4b79-ddeb-4cd8-a64b-7c8234fc4891 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:23.979822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 603/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:23.981818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:24.108922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61246f-f2c1-4814-adf1-0061c556fc0a None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:24.141830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:24.148673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:24.149349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:24.149732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:24.150019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:24.150890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=2aaa83fc8b334b07b67a2ff908a4a105) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:24.151197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:24.152729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dcccc-9e5d-4abd-87ed-f257db5e2330 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:24.159853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 616/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:57:23 2022] GET /identity/v3/role_assignments?user.id=2aaa83fc8b334b07b67a2ff908a4a105&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:26.259781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879415-468a-46a9-b55a-f87dea1a61dc None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:26.260420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879415-468a-46a9-b55a-f87dea1a61dc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:26.260420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879415-468a-46a9-b55a-f87dea1a61dc None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:26.262070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 604/1220] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:26.267345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:26.267873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:26.267873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:26.285303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:26.287143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:26.358394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 617/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:26.381513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879415-468a-46a9-b55a-f87dea1a61dc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:26.496638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879415-468a-46a9-b55a-f87dea1a61dc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:26.536880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:26.537602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:26.537887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:26.538187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:26.549194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:26.557493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:26.558656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:26.559723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 605/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:57:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:26.564812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:26.667093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d13115-3bb7-4b6c-b770-bc3414350dbb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:26.692594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:26.703945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:26.704646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:26.704946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:26.705211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:26.705963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:26.706326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:26.707899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:26.716362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 618/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:57:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:26.722505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:26.768743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:26.830472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c17615-4b7d-4fdf-a88b-8460df555d81 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:26.867392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:26.868204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:26.868489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:26.868763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:26.879830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:26.881592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:26.899255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:26.900387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:26.901311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:26.901311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:26.901311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:26.901311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:26.901311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 606/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:26.906526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.008515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54bc57-6272-4fe7-89fa-15d8c0007ed1 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.045177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.045888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.046153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.046396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.047123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:27.047427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.048824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.061474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 619/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:27.067527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.169558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:27.177830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96785f02-cc02-459b-9942-cb7305db5bb2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.211159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.212091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.212425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.212743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.223743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.225237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:27.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:27.241075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:27.247976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 607/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:57:27.248746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.296642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:27.351820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb46609-d447-450e-9fcb-983a589a5c8b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.386197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.387262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.387554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.387827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.388728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:27.389002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.390407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.409038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 620/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:27.415712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.529781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c3dd7d-44a2-4a78-af39-c351ecf2b1d6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.567955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.568865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.569116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.569397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.580998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.583220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:27.602392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:27.603388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:27.604380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:27.604380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:27.604380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:27.604380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:27.604380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 608/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:27.609787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.722469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111704c-22a5-463a-ba4d-b1d9f1be9bb7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.725376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:27.760191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.760856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.761142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.761681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.762399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:27.762710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.764079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.777837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 621/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:27.783574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:27.795331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:27.892979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f9d30-8555-4ad1-a9d6-b48c13117cd2 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:27.929168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:27.930145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:27.930401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:27.930701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:27.938219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:27.939666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:27.955466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:27.956842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:27.958805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 609/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:27.963544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:28.074161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a05f5-a070-41f9-92e5-ba1bc49a2674 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:28.111248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:28.111917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:28.112229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:28.112512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:28.113316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:28.113627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:28.115066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e632bbfe-0197-41ca-b830-57ba9af25261 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:28.135269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 622/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:28.137066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:28.210087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:28.249390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cfbadd-b87e-4f30-96a4-39cea42be645 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:28.287456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:28.288500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:28.288778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:28.289054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/2aaa83fc8b334b07b67a2ff908a4a105/roles/37c49f81b612403bb86f8edb4539d7af` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:28.307550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=2aaa83fc8b334b07b67a2ff908a4a105, role_id=37c49f81b612403bb86f8edb4539d7af)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:28.309388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd909-2961-4d16-a503-e7624a7e4d71 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:28.326028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 610/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:57:28 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/2aaa83fc8b334b07b67a2ff908a4a105/roles/37c49f81b612403bb86f8edb4539d7af => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:57:30.575241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02a13-77b4-47ea-95b7-e2bb64d14a43 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:30.575806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02a13-77b4-47ea-95b7-e2bb64d14a43 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:30.575983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef02a13-77b4-47ea-95b7-e2bb64d14a43 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:30.577580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 623/1233] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:30.584071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:30.584640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:30.585854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:30.605629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:30.606820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:30.679813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 611/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:30.708673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02a13-77b4-47ea-95b7-e2bb64d14a43 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:30.734988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:30.816186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef02a13-77b4-47ea-95b7-e2bb64d14a43 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:30.862028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:30.862891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:30.863309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:30.863594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:30.877961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:30.886871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:30.887892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:30.893007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 624/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Mon Jun 20 06:57:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:30.894193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:30.950059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:31.013694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d52fd6-4bbf-42d4-9263-5e60d80e19c9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.054932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.056049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.056596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.057088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:31.058511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:31.062306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:31.065015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:31.084872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 612/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Mon Jun 20 06:57:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:31.086135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:31.185871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:31.203671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b51ec-1ee2-4781-888b-837991ccd6da None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.243509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.244411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.244700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.245045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:31.257782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:31.260219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:31.279651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:31.280891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:31.281812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:31.281812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:31.281812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:31.281812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:31.281812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 625/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:31.287515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:31.391633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277e0ac-ba41-459d-9e92-010809616be5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.402275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:31.431116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.431954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.432335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.432620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:31.433416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:31.433741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:31.435280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:31.447501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 613/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:31.454548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:31.572080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452f36c-d933-4965-a00e-2ca1d2c5a74d None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.609483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.610282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.610577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.610844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:31.613678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:31.619117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:31.620715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:31.636031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:31.637327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:31.638386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 626/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:31.643576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:31.768731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a82a40-21bb-45ef-be8e-b2a2637225ec None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.807490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.808187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.808478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.808759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:31.809598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:31.809911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:31.811346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:31.829815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 614/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:31.836237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:31.945197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebe22d-4d70-4f06-9ddd-9dfbbe30d0b4 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:31.983823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:31.984520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:31.984812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:31.985094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:32.057583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:32.065461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:32.067620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:32.180150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 627/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:57:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:57:32.187034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:32.217012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:32.291674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52c306-913f-4646-89f4-b1ea07700fad None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:32.327336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:32.328423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:32.328816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:32.329107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:32.341204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:32.342692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:32.364926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:32.365903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:32.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:32.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:32.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:32.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:32.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 615/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:32.372562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:32.478665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:32.486625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34ce19-c021-41bb-bac3-ba7185d2fbc7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:32.527028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:32.527792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:32.528074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:32.528453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:32.529263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:32.529602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:32.531149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:32.544584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 628/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:32.550199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:32.642523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:32.667863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82b0b3-0cc1-4a20-befd-78934d706250 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:32.709117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:32.710023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:32.710365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:32.710671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:32.719005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:32.721121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:32.736993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:32.738063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:32.739014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 616/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:32.745087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:32.852893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21dc45-58b0-42bb-aa8b-d869528f8067 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:32.888401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:32.889072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:32.889360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:32.889649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:32.890433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:32.890808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:32.892239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0bfce1-71e0-45fa-a156-8d84a0518645 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:32.905450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 629/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:32.912153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:33.018938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9ad0-4d6e-4c8a-a85d-f2ea2254f756 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:33.055864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:33.056713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:33.057068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:33.057355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:33.058327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] RBAC: Adding query filter params (role.id=37c49f81b612403bb86f8edb4539d7af, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=2aaa83fc8b334b07b67a2ff908a4a105) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:33.058718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:33.060359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9f5a5-0e4b-46a0-8eda-4a6a1cbb2067 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:33.068731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 617/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:57:32 2022] GET /identity/v3/role_assignments?user.id=2aaa83fc8b334b07b67a2ff908a4a105&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=37c49f81b612403bb86f8edb4539d7af => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:35.309203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc268325-bb71-4508-90c9-3aca48bae66b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:35.309771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc268325-bb71-4508-90c9-3aca48bae66b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:35.309887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc268325-bb71-4508-90c9-3aca48bae66b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:35.311491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 630/1247] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:57:35.318140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:35.318582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:35.318756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:35.324488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:35.338430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:35.339723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:35.418955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 618/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 06:57:35.443394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc268325-bb71-4508-90c9-3aca48bae66b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:35.462450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:35.562665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc268325-bb71-4508-90c9-3aca48bae66b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:35.600928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:35.601760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:35.602055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:35.602341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:35.613648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:35.621288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:35.622540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:35.623941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 631/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:35.629977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:35.743069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf03f53-9cb3-4215-867f-545b8ed9ed6b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:35.776461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:35.782859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:35.783537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:35.783815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:35.784100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:35.784863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:35.785169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:35.786597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:35.795450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 619/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:57:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:35.801802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:35.892636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:35.921850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3d2d-6eaf-4fd2-8ed8-d8e54fd802b3 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:35.962964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:35.963825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:35.964136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:35.964423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:35.976898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:35.978393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:35.998198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:35.999442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:36.000465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:36.000465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:36.000465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:36.000465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:36.000465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 632/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:36.006437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:36.122006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d968ffe5-0402-448a-b7b4-bf1098811d93 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:36.158841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:36.159429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:36.159713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:36.160532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:36.161258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:36.161664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:36.164559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:36.178934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 620/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:36.184725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:36.304073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbf5a9-ee8e-4f96-aae7-2290b4cc4c30 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:36.334787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:36.344074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:36.345029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:36.345321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:36.345597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:36.355452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:36.357262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:36.374162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:36.375369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:36.376644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 633/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:36.384164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:36.456426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:36.514598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07f8a2-4421-4a8b-b5e5-24f6ae261eaa None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:36.553868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:36.554559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:36.554873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:36.555120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:36.555953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:36.556300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:36.557728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:36.576290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 621/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:36.582940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:36.689658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a974871-5235-449c-8dcc-8b435d9f30cd None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:36.727183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:36.727916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:36.728307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:36.728507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:36.788259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:36.801230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:36.802686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:36.906508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 634/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:57:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:57:36.912227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:37.023450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad16f69-9c4e-43e1-af47-dedfc6525429 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:37.039694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:37.063659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:37.064482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:37.064806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:37.065055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:37.075924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:37.077634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:37.100868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:37.102098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:37.103024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:37.103024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:37.103024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:37.103024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:37.103024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 622/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 06:57:37.107995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:37.205607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:37.218816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaca1-60ca-4985-a48b-c0b3e3850678 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:37.255164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:37.255858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:37.256243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:37.256447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:37.257193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:37.257494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:37.258874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:37.276231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 635/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:37.277149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:37.384457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad47ed-36d0-4ac2-8f77-8a06ee118de3 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:37.426264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:37.427193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:37.427487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:37.427769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:37.436360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:37.437855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:37.452858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:37.453932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:37.454846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 623/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:37.461450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:37.577649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33467-ec4f-4022-b0cd-ac00ca809607 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:37.611823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:37.612515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:37.612827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:37.613132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:37.613926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:37.614239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:37.615682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac469ed-921f-416c-96dd-b63f6fbf8c8c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:37.627899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 636/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:37.634622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:37.663957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:37.746588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fb772-8bae-4f47-a521-5808b9b4366b None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:37.782663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:37.783325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:37.783609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:37.783888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:37.784758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=2aaa83fc8b334b07b67a2ff908a4a105) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:37.785098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:37.786534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f73aa5-7509-4d00-9b52-390ae09d238a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:37.792355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 624/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:57:37 2022] GET /identity/v3/role_assignments?user.id=2aaa83fc8b334b07b67a2ff908a4a105&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:39.809628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198f34-40a0-4487-b745-db809a206250 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:39.810148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198f34-40a0-4487-b745-db809a206250 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:39.810148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a198f34-40a0-4487-b745-db809a206250 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:39.811531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 637/1261] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:39.817110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:39.817484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:39.817681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:39.834702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:39.835884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:39.900984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 625/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:39.922968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a198f34-40a0-4487-b745-db809a206250 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:39.933512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.021083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a198f34-40a0-4487-b745-db809a206250 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.053783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.055089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.055589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.055875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.066946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:40.074438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:40.075546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:40.076824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 638/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:57:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:40.082305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.100717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.179134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306ecc6-bef6-461c-a6bc-7eec17ae8f93 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.211688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.212367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.212652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.212928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.213678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:40.213984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:40.215357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:40.223150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 626/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:57:40.228879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.292825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.325370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3346a60-df7b-4397-8f9e-cbed0042b03b None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.362494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.363807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.364089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.364398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.374647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:40.376093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:40.393349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:40.394443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:40.395723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:40.395723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:40.395723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:40.395723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:40.395723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 639/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:40.400329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.482372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.505791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0845d0-b040-4e92-b61f-73107e48aa0f None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.541976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.542626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.542958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.543185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.543896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:40.544296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:40.545655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:40.557562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 627/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:40.563201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.662240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c65f-9523-40cd-a659-01af86ce07fb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.681449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.695023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.695812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.696093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.696394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.703862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:40.705330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:40.722709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:40.723890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:40.725193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 640/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:40.728278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.824483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4b82-e0eb-46d2-a113-30d5c61b7662 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:40.857366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:40.858018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:40.858303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:40.858577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:40.859398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:40.859708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:40.861153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:40.870038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:40.878366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 628/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:40.883764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:40.979759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5afa54-765d-4771-a363-472fc596c9ee None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:41.011641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:41.012434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:41.012719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:41.012993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:41.023599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:41.025055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:41.042072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:41.043125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:41.044276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:41.044276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:41.044276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:41.044276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:41.044276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 641/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:41.048750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:41.143654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab048d-e26b-4736-9c83-6bc752c40692 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:41.176701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:41.177343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:41.177625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:41.177899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:41.178610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:41.178917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:41.180308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:41.191073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 629/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:41.196762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:41.228185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:41.294281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd6c3a-68e2-4835-9212-e770f9df7288 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:41.326041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:41.326803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:41.327088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:41.327362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:41.334304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:41.335722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:41.349362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:41.350410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:41.351550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 642/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:41.356259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:41.413306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:41.453432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecd4dd-11af-4a73-8c47-015c07faf1d5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:41.490045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:41.490755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:41.491195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:41.491420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:41.492312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:41.492695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:41.494367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333dd552-9368-40bb-b2b7-adc0f4d1adeb None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:41.507903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 630/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:41.514109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:41.601453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:41.610275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0109c1a-cedf-4562-a195-817935af2f05 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:41.643123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:41.644050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:41.644847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:41.645124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] PATH_INFO: `/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/2aaa83fc8b334b07b67a2ff908a4a105/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:41.647039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fe725ff03146febe44bdfa6ff51a78, user_id=2aaa83fc8b334b07b67a2ff908a4a105, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:41.648488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72561556-6b7e-4a52-bcb4-12e5f967eb1a None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:41.660084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 643/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Mon Jun 20 06:57:41 2022] PUT /identity/v3/projects/d1fe725ff03146febe44bdfa6ff51a78/users/2aaa83fc8b334b07b67a2ff908a4a105/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:57:43.644214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508619b-7f91-427b-acef-753741e15c39 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:43.644787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508619b-7f91-427b-acef-753741e15c39 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:43.644787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508619b-7f91-427b-acef-753741e15c39 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:43.646140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 631/1274] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:43.651118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:43.651496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:43.651669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:43.668390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:43.669568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:43.734296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 644/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:43.757808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508619b-7f91-427b-acef-753741e15c39 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:43.856118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508619b-7f91-427b-acef-753741e15c39 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:43.873546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:43.890728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:43.891501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:43.891784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:43.892061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:43.902516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 06:57:43.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:43.911064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:43.912151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 632/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Mon Jun 20 06:57:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:43.916890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:43.995148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:44.013834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c137bac-4002-469a-b14d-954cbb0010f2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.049100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.049100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.049351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.049497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.050786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:44.051091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.052484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.060696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 645/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Mon Jun 20 06:57:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:44.066955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:44.162135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fa443-b510-4fba-a335-a7654e4702f6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.194696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.195457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.195740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.196015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.206171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.207661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:44.224152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:44.225259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:44.226386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:44.226386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:44.226386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:44.226386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:44.226386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 633/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:44.230906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:44.332436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081df9c9-1cfa-4bd1-a5b0-4d827afb23c7 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.365217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.365864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.366147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.366426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.367137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:44.367442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.368830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.380202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 646/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:44.385700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:44.416756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:44.492853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e85c0c-7fd1-40f4-9469-8e4afc457ff7 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.525987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.526770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.527052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.527326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.535012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.536469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 06:57:44.550747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:44.551893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:44.552946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 634/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:44.557737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:44.561493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:44.654852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634e1a7-4c25-4274-bd82-d4dcb339a5aa None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.688351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.688947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.689225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.689496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.690312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:44.690618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.692036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.709086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 647/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Mon Jun 20 06:57:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:44.714721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:44.803961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:44.813621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a7c6a-fb88-49da-8051-e4dc253e4c8d None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:44.845571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:44.846196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:44.846477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:44.846754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:44.911282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:44.912736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:44.996325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:45.006701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 635/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Mon Jun 20 06:57:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 06:57:45.012722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:45.075230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:45.112445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5f35a-e8d2-45cc-9510-ef1c2d91eab2 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:45.145527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:45.146296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:45.146578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:45.146854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:45.156513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:45.157944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 06:57:45.174725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:45.175871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:45.177055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 06:57:45.177055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 06:57:45.177055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 06:57:45.177055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:45.177055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 648/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 06:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:45.181552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:45.278870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479bf22-f65a-4d38-9942-5a5dafe6d23a None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:45.311378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:45.312029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:45.312343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:45.312621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:45.313337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:45.313644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:45.315005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:45.326525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 636/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Mon Jun 20 06:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:45.332761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:45.432257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e424ade-6ebf-42a2-b62c-24129c7cab50 None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:45.447012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:45.469126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:45.469916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:45.470194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:45.470467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:45.477535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:45.479143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 06:57:45.493304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:45.494390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: ERROR keystone.server.flask.application Jun 20 06:57:45.495693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 649/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Mon Jun 20 06:57:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:45.500278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:45.549270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:45.597824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30825fbf-78f9-423d-bc93-b36704e42ee6 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:45.630745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:45.631393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:45.631682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:45.631963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:45.632777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:45.633089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:45.634518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2ae5e-8a5d-46fd-81ba-91ce68c6349c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:45.646442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 637/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Mon Jun 20 06:57:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:45.652700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:45.749376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce837260-9bba-4e30-a6e7-d9a9dae9fbbb None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:45.781795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:45.782434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:45.782716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:45.782993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:45.783817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] RBAC: Adding query filter params (role.id=53acd1fbd8c545949ec0b81266c9d3ad, scope.project.id=d1fe725ff03146febe44bdfa6ff51a78, user.id=2aaa83fc8b334b07b67a2ff908a4a105) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:45.784153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:45.786116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b537e-6882-45e6-bb79-8a4a0b29db84 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:45.791874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 650/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Mon Jun 20 06:57:45 2022] GET /identity/v3/role_assignments?user.id=2aaa83fc8b334b07b67a2ff908a4a105&scope.project.id=d1fe725ff03146febe44bdfa6ff51a78&role.id=53acd1fbd8c545949ec0b81266c9d3ad => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:47.839533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372b793-0967-426c-b47d-bdb567a5f56c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:47.840096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372b793-0967-426c-b47d-bdb567a5f56c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:47.840096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372b793-0967-426c-b47d-bdb567a5f56c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:47.841468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 638/1288] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:47.846557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:47.847530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:47.847729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:47.864064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:47.865286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:47.894185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:47.930615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 651/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:47.952388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372b793-0967-426c-b47d-bdb567a5f56c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:48.051938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372b793-0967-426c-b47d-bdb567a5f56c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:48.062216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:48.086631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:48.087415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:48.087700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:48.087979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:48.088672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:48.090038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:48.101660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:48.102996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:48.104020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 639/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:57:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:48.108687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:48.204573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796175-537e-4c18-a56c-5607079abc01 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:48.237087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:48.238338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:48.238623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:48.238902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:48.239605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:48.239915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:48.241318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64a210-9bee-4a45-8506-1aaeffe20874 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:48.245973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 652/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Jun 20 06:57:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:48.251375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:48.347680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3315831-645b-4a7d-967d-c2e132366763 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:48.379813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:48.380498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:48.380825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:48.381147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] PATH_INFO: `/v3/services` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:48.381881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:48.382211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:48.383565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcbd4b-c7fc-4100-8cbc-638cad74801d None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:48.388096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 640/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Jun 20 06:57:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:50.411135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9bd8f-4a7d-4502-a1e5-33b3981a125f None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:50.411661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9bd8f-4a7d-4502-a1e5-33b3981a125f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:50.411661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9bd8f-4a7d-4502-a1e5-33b3981a125f None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:50.413080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 653/1293] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:50.418189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367b99-ad1b-4dee-8311-98b07dd75284 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:50.418605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367b99-ad1b-4dee-8311-98b07dd75284 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:50.418838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a367b99-ad1b-4dee-8311-98b07dd75284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:50.436011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9a367b99-ad1b-4dee-8311-98b07dd75284 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:50.437225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9a367b99-ad1b-4dee-8311-98b07dd75284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:50.468218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:50.513191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 641/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:50.540122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9bd8f-4a7d-4502-a1e5-33b3981a125f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:50.572297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:50.640155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9bd8f-4a7d-4502-a1e5-33b3981a125f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:50.672527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:50.673110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:50.673392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:50.673666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:50.674377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:50.676314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b111e0-ec89-44d8-9b40-79fc7e0529b7 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:50.690794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 654/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Mon Jun 20 06:57:50 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:57:52.685172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bf88f-9478-4b98-8d8f-5a55dedb25e3 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:52.685737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bf88f-9478-4b98-8d8f-5a55dedb25e3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:52.685840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1bf88f-9478-4b98-8d8f-5a55dedb25e3 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:52.687231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 642/1296] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:52.695650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:52.695650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:52.695650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:52.711227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:52.712451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:52.782418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 655/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:52.804689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bf88f-9478-4b98-8d8f-5a55dedb25e3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:52.904296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1bf88f-9478-4b98-8d8f-5a55dedb25e3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:52.937477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:52.938197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:52.938482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:52.938760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:52.939411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:52.941397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:52.952735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:52.953962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:52.955119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 643/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:57:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:52.959578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:52.977143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:53.054796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7c174-8090-4090-b2d7-ac4c1e932a02 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:53.087713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:53.088388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:53.088671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:53.088947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:53.089648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] RBAC: Adding query filter params (name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:53.089957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:53.091321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:53.096300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 656/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Jun 20 06:57:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:53.102112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:53.133498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:53.199040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418140-174d-4e87-b94a-723d24138a5c None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:53.230507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:53.231138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:53.231419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:53.231694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:53.232508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] RBAC: Adding query filter params (interface=public, service_id=1d22c062f38c4d698e0196ea77f658a0, region_id=RegionOne) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:53.232816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:53.234162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea070cd-75e3-4f36-9ade-89e9851667a7 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:53.239590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 644/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Mon Jun 20 06:57:53 2022] GET /identity/v3/endpoints?service_id=1d22c062f38c4d698e0196ea77f658a0&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:53.245105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:53.323899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:53.341661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c13e68-2d2b-49b9-afaf-2a30e62fc7da None admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:53.374665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:53.375313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:53.375597 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:53.375874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:53.376581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:53.377947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296228-5c34-4757-8865-ecc2ff94e08c None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:53.384579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 657/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Mon Jun 20 06:57:53 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:57:55.367035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529153b-d57b-48db-9ec5-75de32e56835 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:55.367575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529153b-d57b-48db-9ec5-75de32e56835 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:55.367669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529153b-d57b-48db-9ec5-75de32e56835 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:55.369040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 645/1302] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:57:55.374709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:55.375143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:55.375254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:55.394808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:57:55.396005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:57:55.460395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 658/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:57:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 06:57:55.485246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529153b-d57b-48db-9ec5-75de32e56835 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:55.582130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:55.589149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b529153b-d57b-48db-9ec5-75de32e56835 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:55.622293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:55.623015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:55.623297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:55.623572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:55.624243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:55.625615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:57:55.637328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:57:55.643380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application with Lock( Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: ERROR keystone.server.flask.application Jun 20 06:57:55.644567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 646/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Mon Jun 20 06:57:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 06:57:55.645604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:55.731610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:57:55.754754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd67960-c7cc-4f0c-ad74-2ef80ac5525d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:55.787066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:55.787711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:55.787991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:55.788294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] PATH_INFO: `/v3/services` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:55.788991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:57:55.789294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:55.790645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a728d-4a56-472b-9087-3443960a5b45 None admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:55.795594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 659/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Mon Jun 20 06:57:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:57:55.802050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:57:55.898315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b6961-272f-4a64-81e4-4d2ffef7cfd5 None admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:57:55.931177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:57:55.931812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:57:55.932102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:57:55.932404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:57:55.933145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:57:55.934496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b3773-3d2b-4cb3-a63f-53e1dc3a49d5 None admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:57:55.952399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 647/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Mon Jun 20 06:57:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:58:05.792798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=30 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:58:05.948381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:58:58.129584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54c3a-0951-4c83-b857-1bdeebd4b548 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:58:58.130047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54c3a-0951-4c83-b857-1bdeebd4b548 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:58:58.130128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f54c3a-0951-4c83-b857-1bdeebd4b548 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:58:58.136957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da3df-df8c-48f9-a7a7-13fdfb0bcefc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:58:58.137359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da3df-df8c-48f9-a7a7-13fdfb0bcefc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:58:58.137504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14da3df-df8c-48f9-a7a7-13fdfb0bcefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:58:58.140473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 660/1307] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:58:58 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:58:58.299745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b14da3df-df8c-48f9-a7a7-13fdfb0bcefc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:58:58.302293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b14da3df-df8c-48f9-a7a7-13fdfb0bcefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:58:58.456706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 648/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:58:58.506263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:58:58.506547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:58:58.507016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:58:58.508442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 661/1309] 10.222.0.23 () {54 vars in 964 bytes} [Mon Jun 20 06:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:58:58.515959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541f8b-eed1-4e03-8bc5-10e62a7db458 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:58:58.516296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541f8b-eed1-4e03-8bc5-10e62a7db458 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:58:58.516576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb541f8b-eed1-4e03-8bc5-10e62a7db458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:58:58.596318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cb541f8b-eed1-4e03-8bc5-10e62a7db458 None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:58:58.597526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cb541f8b-eed1-4e03-8bc5-10e62a7db458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:58:58.618781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:58:58.677853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 649/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Mon Jun 20 06:58:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:58:58.686532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:58:58.688797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:58:58.807725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae3e4-f8f5-40fd-b59d-abe0d3b1aa57 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:58:58.883762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:58:58.884436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:58:58.884715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:58:58.885242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:58:58.886849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:58:58.943680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:58:59.001558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:58:59.002989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50b6b6-bb0d-4ba4-b930-70f3f2a7a639 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:58:59.023178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 662/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:01.337543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57651-fa16-464f-be7a-51b3aa907691 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:01.338089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57651-fa16-464f-be7a-51b3aa907691 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:01.338089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc57651-fa16-464f-be7a-51b3aa907691 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:01.339621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 650/1312] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:01.345297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deeb5-6d13-48a5-8cde-6b2fc74a64a3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:01.345708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deeb5-6d13-48a5-8cde-6b2fc74a64a3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:01.345924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6deeb5-6d13-48a5-8cde-6b2fc74a64a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:01.430680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0a6deeb5-6d13-48a5-8cde-6b2fc74a64a3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:01.432175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0a6deeb5-6d13-48a5-8cde-6b2fc74a64a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:01.543241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 663/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:01.598603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:01.598790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:01.599072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:01.601034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 651/1314] 10.222.0.23 () {54 vars in 964 bytes} [Mon Jun 20 06:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:01.607330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221eee2-8dc0-4d6f-834e-a4cec1f4ca30 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:01.607671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221eee2-8dc0-4d6f-834e-a4cec1f4ca30 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:01.607891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221eee2-8dc0-4d6f-834e-a4cec1f4ca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:01.618819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:01.694399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f221eee2-8dc0-4d6f-834e-a4cec1f4ca30 None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:01.694399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f221eee2-8dc0-4d6f-834e-a4cec1f4ca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:01.784960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 664/1315] 10.222.0.23 () {58 vars in 1061 bytes} [Mon Jun 20 06:59:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:01.794009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:01.795517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:01.916312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45140cd-e923-4921-b881-c4e0170a4de5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:01.986977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:01.989792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:01.990651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:01.991133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:01.991607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:01.993299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:02.101796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:02.103147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13583-6de6-48f4-8940-d4d44c58db33 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:02.121732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 652/1316] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:59:04.661905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:04.662440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:04.662539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:04.663853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 665/1317] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:04.669082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef154d0-bb1c-4cfb-a8e8-0ba1991d9d65 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:04.669516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef154d0-bb1c-4cfb-a8e8-0ba1991d9d65 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:04.669645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef154d0-bb1c-4cfb-a8e8-0ba1991d9d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:04.734810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0ef154d0-bb1c-4cfb-a8e8-0ba1991d9d65 None None] MFA Rules not processed for user `abeac7d1d42343afbdb5c2951af21e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:04.736025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0ef154d0-bb1c-4cfb-a8e8-0ba1991d9d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:04.787698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:04.832628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 653/1318] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:04.871862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:04.873397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:04.974091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6660cb3-3be3-446e-93c2-7e87e1150fbf None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:04.977106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:05.046795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:05.047425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:05.047711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:05.047990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:05.049656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:05.153842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:05.155245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1a729-2360-402e-a84e-edd75bfd89ae service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:05.172880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 666/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:07.721844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:07.722309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:07.722404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:07.723828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 654/1320] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:59:07.728970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e1e3-6d31-4217-86f3-6d7ae7a409bb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:07.729385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e1e3-6d31-4217-86f3-6d7ae7a409bb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:07.729529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a7e1e3-6d31-4217-86f3-6d7ae7a409bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:07.768739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:07.794032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-43a7e1e3-6d31-4217-86f3-6d7ae7a409bb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:07.795227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-43a7e1e3-6d31-4217-86f3-6d7ae7a409bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:07.888847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 667/1321] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:07.928201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:07.929719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:08.032715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e654f2-541c-4523-a97a-90c7d3e563ff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:08.103103 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:08.103738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:08.104024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:08.104330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:08.105949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:08.113205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:08.212783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:08.214249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2be44b-7403-4412-9c7a-7bf695621e2e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:08.232437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 655/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:11.145802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598a67-01de-4054-95f7-604782dca3ee None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:11.146325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598a67-01de-4054-95f7-604782dca3ee None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:11.146416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48598a67-01de-4054-95f7-604782dca3ee None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:11.147749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 668/1323] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:11.153003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67971c9b-ee46-4fc2-b3f7-8a970f3e924c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:11.153437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67971c9b-ee46-4fc2-b3f7-8a970f3e924c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:11.153565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67971c9b-ee46-4fc2-b3f7-8a970f3e924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:11.217630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-67971c9b-ee46-4fc2-b3f7-8a970f3e924c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:11.219458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-67971c9b-ee46-4fc2-b3f7-8a970f3e924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:11.284222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:11.326000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 656/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:11.362706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48598a67-01de-4054-95f7-604782dca3ee None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:11.364320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-48598a67-01de-4054-95f7-604782dca3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:11.472561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48598a67-01de-4054-95f7-604782dca3ee None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:11.485218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:11.546145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:11.546800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:11.547084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:11.547364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:11.549037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:11.655262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:11.656659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cc19f-7d42-40af-820b-2a10dfd2a546 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:11.675060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 669/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:13.762093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:13.762614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:13.762731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:13.764015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 657/1326] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:13.769162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbe59-5c3f-4b2c-98dc-b6115f792bb3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:13.769546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbe59-5c3f-4b2c-98dc-b6115f792bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:13.769801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbe59-5c3f-4b2c-98dc-b6115f792bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:13.816168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:13.834758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e5dbbe59-5c3f-4b2c-98dc-b6115f792bb3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:13.835955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e5dbbe59-5c3f-4b2c-98dc-b6115f792bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:13.932553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 670/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:13.968514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:13.970031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:14.071719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb11503-df14-4bbd-8027-5d6a5ec8163e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:14.148411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:14.149048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:14.149328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:14.149600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:14.151205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:14.168272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:14.258476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:14.260220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7c6a2-49c2-4fca-bbab-dd80e6814313 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:14.278746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 658/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:16.360102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:16.360668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:16.360668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:16.362087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 671/1329] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:16.367267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71417-742d-403f-a8e6-549cfb6ac135 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:16.367650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71417-742d-403f-a8e6-549cfb6ac135 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:16.367940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe71417-742d-403f-a8e6-549cfb6ac135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:16.435041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cbe71417-742d-403f-a8e6-549cfb6ac135 None None] MFA Rules not processed for user `abeac7d1d42343afbdb5c2951af21e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:16.436306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cbe71417-742d-403f-a8e6-549cfb6ac135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:16.504424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:16.546395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 659/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:16.583894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:16.585434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:16.687025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3237f5-c56c-45d3-9be8-25edf29f1614 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:16.707187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:16.762685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:16.763331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:16.763608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:16.763880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:16.765551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:16.872237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:16.873663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa346c82-d473-433f-a290-48566401a663 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:16.894709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 672/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:19.190725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:19.191278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:19.191278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:19.192700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 660/1332] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:19.197759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d158-9777-4755-acf9-c34785f588e2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:19.198192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d158-9777-4755-acf9-c34785f588e2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:19.198319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b3d158-9777-4755-acf9-c34785f588e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:19.258357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:19.261463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-96b3d158-9777-4755-acf9-c34785f588e2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:19.263253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-96b3d158-9777-4755-acf9-c34785f588e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:19.355423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 673/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:19.391069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:19.392603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:19.495327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd66f27-4eb2-4871-b020-5ead566984f9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:19.574970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:19.575609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:19.575888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:19.576185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:19.577828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:19.604053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:19.685656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:19.687046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49967007-244a-4447-b334-4c9059d943ab service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:19.705671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 661/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:21.776905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:21.777446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:21.777446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:21.778845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 674/1335] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:21.784487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cea0-58f9-49ec-aaf4-c80d81cbd345 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:21.784906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cea0-58f9-49ec-aaf4-c80d81cbd345 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:21.785056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622cea0-58f9-49ec-aaf4-c80d81cbd345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:21.849429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8622cea0-58f9-49ec-aaf4-c80d81cbd345 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:21.850630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8622cea0-58f9-49ec-aaf4-c80d81cbd345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:21.928888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:21.950829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 662/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:21.990589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:21.992125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:22.095054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f99a575-d4e4-4005-8fd8-78e4ba4da698 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:22.122668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:22.166641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:22.167277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:22.167558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:22.167831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:22.169486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:22.284378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:22.285824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a718303-c351-4119-8a5e-e4974b1a1c32 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:22.303180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 675/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:24.700384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:24.700922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:24.700922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:24.702339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 663/1338] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:24.707833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb315a-dc35-42a6-b591-cb7229fc69fb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:24.708236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb315a-dc35-42a6-b591-cb7229fc69fb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:24.708427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb315a-dc35-42a6-b591-cb7229fc69fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:24.772088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a3fb315a-dc35-42a6-b591-cb7229fc69fb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:24.773314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a3fb315a-dc35-42a6-b591-cb7229fc69fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:24.781947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:24.869606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 676/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:24.909024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:24.910567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:25.012870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4f0aa-f5bd-44d2-a040-e332505f7cf2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:25.095757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:25.096426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:25.096714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:25.096997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:25.098644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:25.137052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:25.210378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:25.211827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ee33c-8a91-414d-b940-fe957a2f64f6 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:25.230054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 664/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:29.286776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:29.287332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:29.287332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:29.288729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 677/1341] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:29.294018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1f42-bc4e-4312-a276-37ef9a51bee4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:29.294432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1f42-bc4e-4312-a276-37ef9a51bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:29.294594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db1f42-bc4e-4312-a276-37ef9a51bee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:29.365197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-15db1f42-bc4e-4312-a276-37ef9a51bee4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:29.366429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-15db1f42-bc4e-4312-a276-37ef9a51bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:29.458747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:29.472374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 665/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:29.512458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:29.515185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:29.611739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5463adb-56be-43f7-830f-38fa1a72a7e7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:29.615301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:29.615937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:29.616252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:29.616531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:29.626019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:29.627426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c2ee-b4ed-4f41-90b3-282c5c5f8a32 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:29.637011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 678/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:32.284122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb63ea-c3fc-4373-a9ef-626da691b774 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:32.284658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb63ea-c3fc-4373-a9ef-626da691b774 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:32.284773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb63ea-c3fc-4373-a9ef-626da691b774 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:32.286055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 666/1344] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:32.291176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0cd88-2cd2-47c9-bf6b-d9bfcdd7132c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:32.291588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0cd88-2cd2-47c9-bf6b-d9bfcdd7132c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:32.291731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0cd88-2cd2-47c9-bf6b-d9bfcdd7132c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:32.320544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c5b0cd88-2cd2-47c9-bf6b-d9bfcdd7132c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:32.321726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c5b0cd88-2cd2-47c9-bf6b-d9bfcdd7132c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:32.350202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:32.403333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 679/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:32.441381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb63ea-c3fc-4373-a9ef-626da691b774 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:32.453088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb63ea-c3fc-4373-a9ef-626da691b774 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:32.456591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:32.457247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:32.457527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:32.457807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:32.467750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:32.469299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407647ee-e2d3-47cc-8235-f7a72978c8b5 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:32.481599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 667/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:36.536738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:36.537274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:36.537366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:36.538775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 680/1347] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:36.544124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040703d9-a802-4448-b48d-ac4915fc225d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:36.544567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040703d9-a802-4448-b48d-ac4915fc225d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:36.544682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040703d9-a802-4448-b48d-ac4915fc225d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:36.562293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-040703d9-a802-4448-b48d-ac4915fc225d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:36.563506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-040703d9-a802-4448-b48d-ac4915fc225d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:36.571443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 668/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:36.609908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:36.611653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:36.624426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60d5377-9c21-449f-9d40-1785cf1c5bdb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:36.628255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:36.628926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:36.629193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:36.629466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:36.639596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:36.641051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421d48a-1c54-46f9-bc26-6390222dfd84 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:36.650353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 681/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:41.710598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12088-3683-4600-87f7-533e7e7a32d6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:41.710598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12088-3683-4600-87f7-533e7e7a32d6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:41.710598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff12088-3683-4600-87f7-533e7e7a32d6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:41.710598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 669/1350] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 06:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:41.715295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c52f-743a-4f3c-8f16-831dab006c60 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:41.716261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c52f-743a-4f3c-8f16-831dab006c60 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:41.717319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82c52f-743a-4f3c-8f16-831dab006c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:41.791134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bb82c52f-743a-4f3c-8f16-831dab006c60 None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:41.792352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bb82c52f-743a-4f3c-8f16-831dab006c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:41.878133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 682/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 06:59:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:46.030246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407a1d-b64f-4a6f-b658-7bc9fabeaafd None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:46.030768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407a1d-b64f-4a6f-b658-7bc9fabeaafd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:46.030960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db407a1d-b64f-4a6f-b658-7bc9fabeaafd None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:46.032447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 670/1352] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 06:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:46.038617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cda91-fc4a-472e-8df9-9e226e967e03 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:46.039033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cda91-fc4a-472e-8df9-9e226e967e03 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:46.039176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cda91-fc4a-472e-8df9-9e226e967e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:46.058622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-492cda91-fc4a-472e-8df9-9e226e967e03 None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:46.059906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-492cda91-fc4a-472e-8df9-9e226e967e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:46.069162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 683/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 06:59:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:46.557149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=13 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:47.533284 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2cbb-be9e-4e94-bd80-ca2bcefbcfee None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:47.533938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2cbb-be9e-4e94-bd80-ca2bcefbcfee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:47.533938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2cbb-be9e-4e94-bd80-ca2bcefbcfee None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:47.535223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 671/1354] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 06:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:47.540193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfef76-2898-4af1-be62-8f71c48217ad None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:47.540664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfef76-2898-4af1-be62-8f71c48217ad None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:47.540762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfef76-2898-4af1-be62-8f71c48217ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:47.558668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aecfef76-2898-4af1-be62-8f71c48217ad None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:47.559869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aecfef76-2898-4af1-be62-8f71c48217ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:47.570998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 684/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 06:59:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:51.100850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf3a1-cabb-490c-b209-9811ecd9fe49 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:51.101851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf3a1-cabb-490c-b209-9811ecd9fe49 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:51.102356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbcf3a1-cabb-490c-b209-9811ecd9fe49 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:51.104322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 672/1356] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:51.110682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903f12-df3c-424a-9084-14cbb56233eb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:51.111160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903f12-df3c-424a-9084-14cbb56233eb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:51.111242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9903f12-df3c-424a-9084-14cbb56233eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:51.129502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a9903f12-df3c-424a-9084-14cbb56233eb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:51.131033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a9903f12-df3c-424a-9084-14cbb56233eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:51.140543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 685/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:51.197320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded48d2-b095-445a-9fe4-5d843abc4afb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:51.197667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded48d2-b095-445a-9fe4-5d843abc4afb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:51.197952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded48d2-b095-445a-9fe4-5d843abc4afb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:51.199346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 673/1358] 10.222.0.23 () {54 vars in 961 bytes} [Mon Jun 20 06:59:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:51.208995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bea6ed-647a-460a-a088-fdd0a29df22b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:51.209405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bea6ed-647a-460a-a088-fdd0a29df22b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:51.210500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bea6ed-647a-460a-a088-fdd0a29df22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:51.257824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:59:51.267434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-57bea6ed-647a-460a-a088-fdd0a29df22b None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:51.268757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-57bea6ed-647a-460a-a088-fdd0a29df22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:51.354835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 686/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Jun 20 06:59:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:51.363543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded48d2-b095-445a-9fe4-5d843abc4afb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:51.377939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded48d2-b095-445a-9fe4-5d843abc4afb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:51.383444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:51.384452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:51.384819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:51.385173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:51.394072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:51.395790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18152590-3eac-4fed-8b0d-df3c3f08b311 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:51.406483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 674/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:53.666025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a113e-3e2d-43d1-befa-c9d2962866b8 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:53.666582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a113e-3e2d-43d1-befa-c9d2962866b8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:53.666582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a113e-3e2d-43d1-befa-c9d2962866b8 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:53.667949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 687/1361] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:53.673769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11963-fad2-4dbc-a7e6-effb4608a90a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:53.674198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11963-fad2-4dbc-a7e6-effb4608a90a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:53.674641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11963-fad2-4dbc-a7e6-effb4608a90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:53.693131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-12b11963-fad2-4dbc-a7e6-effb4608a90a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:53.694571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-12b11963-fad2-4dbc-a7e6-effb4608a90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:53.709127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 675/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:53.747167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a113e-3e2d-43d1-befa-c9d2962866b8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:53.761547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a113e-3e2d-43d1-befa-c9d2962866b8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:53.767419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:53.768059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:53.768370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:53.768647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:53.777065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:53.779216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fb311-a516-4482-8a38-9e68cfead99b service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:53.791495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 688/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:56.162800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e9a3-f792-44f4-a09a-4d303a1e633a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:56.163324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e9a3-f792-44f4-a09a-4d303a1e633a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:56.163324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3e9a3-f792-44f4-a09a-4d303a1e633a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:56.164682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 676/1364] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:56.170511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf4a39-40e0-4340-8f8c-f1ea915bf19d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:56.170917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf4a39-40e0-4340-8f8c-f1ea915bf19d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:56.171158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf4a39-40e0-4340-8f8c-f1ea915bf19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:56.189399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-59bf4a39-40e0-4340-8f8c-f1ea915bf19d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:56.190591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-59bf4a39-40e0-4340-8f8c-f1ea915bf19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:56.198757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 689/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:56.244043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbaa80-98eb-4c70-b026-817b2a1868af None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:56.244367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbaa80-98eb-4c70-b026-817b2a1868af None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:56.244651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbaa80-98eb-4c70-b026-817b2a1868af None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:56.246019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 677/1366] 10.222.0.23 () {54 vars in 961 bytes} [Mon Jun 20 06:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:56.251725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f403f67-faee-466b-836f-f522975fe97f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:56.252020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f403f67-faee-466b-836f-f522975fe97f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:56.252335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f403f67-faee-466b-836f-f522975fe97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:56.271069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0f403f67-faee-466b-836f-f522975fe97f None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:56.272291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0f403f67-faee-466b-836f-f522975fe97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:56.281930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 690/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Jun 20 06:59:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:56.289697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbaa80-98eb-4c70-b026-817b2a1868af None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:56.304533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbaa80-98eb-4c70-b026-817b2a1868af None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:56.310098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:56.310740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:56.311027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:56.311311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:56.319619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:56.321237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58369ade-5435-4eba-84bb-c1e3fa235749 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:56.330766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 678/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:59:58.414110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820a40-c665-44e4-aa79-4b7b3f18d086 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:58.414674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820a40-c665-44e4-aa79-4b7b3f18d086 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:58.414674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e820a40-c665-44e4-aa79-4b7b3f18d086 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:58.416101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 691/1369] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:58.422660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89436fc-3fc5-4bcb-b5fc-3301d0b3c89d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:58.423234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89436fc-3fc5-4bcb-b5fc-3301d0b3c89d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:58.423412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89436fc-3fc5-4bcb-b5fc-3301d0b3c89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:58.445089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f89436fc-3fc5-4bcb-b5fc-3301d0b3c89d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:58.446642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f89436fc-3fc5-4bcb-b5fc-3301d0b3c89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:58.455451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 679/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:58.503730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820a40-c665-44e4-aa79-4b7b3f18d086 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:58.519934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e820a40-c665-44e4-aa79-4b7b3f18d086 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:58.525831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:58.526729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:58.527160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:58.527589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:58.537391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:58.539268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c856f7-2d2c-4d73-9f38-b946d0aa1571 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:58.550490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 692/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:59:59.122844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bf30-47f5-45c1-b1bb-9166e1dc04b8 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:59.123245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bf30-47f5-45c1-b1bb-9166e1dc04b8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:59.123455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2bf30-47f5-45c1-b1bb-9166e1dc04b8 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:59.125070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 680/1372] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 06:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:59.130447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616638-06cf-411f-a184-19d6057f17e8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:59.130808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616638-06cf-411f-a184-19d6057f17e8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:59.131159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22616638-06cf-411f-a184-19d6057f17e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:59.149966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-22616638-06cf-411f-a184-19d6057f17e8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:59.151253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-22616638-06cf-411f-a184-19d6057f17e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:59.159973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 693/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 06:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:59.503779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41536199-ec9c-4f77-b980-ed4d25ab3638 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:59.504275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41536199-ec9c-4f77-b980-ed4d25ab3638 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:59.504666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41536199-ec9c-4f77-b980-ed4d25ab3638 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:59.506915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 681/1374] 10.222.0.23 () {54 vars in 963 bytes} [Mon Jun 20 06:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:59:59.512257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0077-6b78-4424-b9de-868beb689673 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:59.513241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0077-6b78-4424-b9de-868beb689673 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:59.513453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf0077-6b78-4424-b9de-868beb689673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:59.562886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6cdf0077-6b78-4424-b9de-868beb689673 None None] MFA Rules not processed for user `468b120f57fe4807b969c390cbcabfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:59:59.564266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6cdf0077-6b78-4424-b9de-868beb689673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:59:59.628013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 694/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Mon Jun 20 06:59:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:59:59.635670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41536199-ec9c-4f77-b980-ed4d25ab3638 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:59:59.646939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41536199-ec9c-4f77-b980-ed4d25ab3638 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:59:59.650915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:59:59.651685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:59:59.652084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:59:59.652499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:59:59.662601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:59:59.664236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08411947-b579-4788-82aa-cfb5228e2a5d service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:59:59.675407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 682/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 06:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:00:00.647146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71710ac-ac60-43c6-8e3a-0de206136de6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:00.647656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71710ac-ac60-43c6-8e3a-0de206136de6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:00.647656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71710ac-ac60-43c6-8e3a-0de206136de6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:00.649290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 695/1377] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:00.654336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e187e82-9ac6-4b03-b1ae-d0f5035544e9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:00.654771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e187e82-9ac6-4b03-b1ae-d0f5035544e9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:00.654902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e187e82-9ac6-4b03-b1ae-d0f5035544e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:00.673762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6e187e82-9ac6-4b03-b1ae-d0f5035544e9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:00.674965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e187e82-9ac6-4b03-b1ae-d0f5035544e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:00.683021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 683/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:00.726682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71710ac-ac60-43c6-8e3a-0de206136de6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:00.741392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71710ac-ac60-43c6-8e3a-0de206136de6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:00.747542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:00.748213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:00.748503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:00.748780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:00.757005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:00.758590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a41ef0-a6c0-44ce-86f3-68445ca1e221 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:00.764566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:00:00.771952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 696/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:02.865760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166e5b-559b-4769-8e8c-4ad992c49b76 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:02.866304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166e5b-559b-4769-8e8c-4ad992c49b76 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:02.866412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9166e5b-559b-4769-8e8c-4ad992c49b76 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:02.867722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 684/1380] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:02.873107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633dcb-33e4-40ad-80b3-f69dc232c267 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:02.873537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633dcb-33e4-40ad-80b3-f69dc232c267 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:02.873743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f633dcb-33e4-40ad-80b3-f69dc232c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:02.892705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9f633dcb-33e4-40ad-80b3-f69dc232c267 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:02.893890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9f633dcb-33e4-40ad-80b3-f69dc232c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:02.901612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 697/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:02.946578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9166e5b-559b-4769-8e8c-4ad992c49b76 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:02.962230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9166e5b-559b-4769-8e8c-4ad992c49b76 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:02.968823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:02.969466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:02.969819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:02.970070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:02.978199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:02.979785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce94dd-516e-4ce8-9ab5-8a14205aef25 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:02.989475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 685/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:05.085752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820c4a-2354-4ffb-a356-d56006b790e9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:05.086296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820c4a-2354-4ffb-a356-d56006b790e9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:05.086296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17820c4a-2354-4ffb-a356-d56006b790e9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:05.087657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 698/1383] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 07:00:05.093037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e14f-28d8-44b8-a08a-c3a9e9596187 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:05.093488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e14f-28d8-44b8-a08a-c3a9e9596187 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:05.093905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496e14f-28d8-44b8-a08a-c3a9e9596187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:05.113450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4496e14f-28d8-44b8-a08a-c3a9e9596187 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:05.114760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4496e14f-28d8-44b8-a08a-c3a9e9596187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:05.123645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 686/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:05.165028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17820c4a-2354-4ffb-a356-d56006b790e9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:05.181366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17820c4a-2354-4ffb-a356-d56006b790e9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:05.189430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:05.190115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:05.190603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:05.190889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:05.200855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:05.202533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08fb6c-c8fd-435c-bb92-559486aa5d29 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:05.214148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 699/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:07.263927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716fc4-910e-4425-b92f-a81dfe2e1573 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:07.264680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716fc4-910e-4425-b92f-a81dfe2e1573 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:07.264680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74716fc4-910e-4425-b92f-a81dfe2e1573 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:07.265886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 687/1386] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:07.271351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808c9d-5745-4ba7-81fe-b2d5e1c5debf None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:07.271910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808c9d-5745-4ba7-81fe-b2d5e1c5debf None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:07.271910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808c9d-5745-4ba7-81fe-b2d5e1c5debf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:07.290318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-15808c9d-5745-4ba7-81fe-b2d5e1c5debf None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:07.291612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-15808c9d-5745-4ba7-81fe-b2d5e1c5debf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:07.299851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 700/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:07.348544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74716fc4-910e-4425-b92f-a81dfe2e1573 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:07.363062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74716fc4-910e-4425-b92f-a81dfe2e1573 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:07.369021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:07.369667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:07.369952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:07.370227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:07.379178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:07.380830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12a245-fbe4-4a9a-9a9d-4807e04925d7 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:07.391323 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 688/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:00:09.520615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40e27-0f73-46b6-b106-dabf49239e30 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:09.521274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40e27-0f73-46b6-b106-dabf49239e30 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:09.521274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade40e27-0f73-46b6-b106-dabf49239e30 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:09.522623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 701/1389] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:09.528443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe9e8-fd70-4d8e-a77a-2f5e13a991d6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:09.528907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe9e8-fd70-4d8e-a77a-2f5e13a991d6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:09.529153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe9e8-fd70-4d8e-a77a-2f5e13a991d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:09.551480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6b0fe9e8-fd70-4d8e-a77a-2f5e13a991d6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:09.553085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6b0fe9e8-fd70-4d8e-a77a-2f5e13a991d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:09.563725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 689/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:09.608352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade40e27-0f73-46b6-b106-dabf49239e30 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:09.623432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade40e27-0f73-46b6-b106-dabf49239e30 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:09.628831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:09.629489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:09.629757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:09.630035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:09.638684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:09.640295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca84085-801d-45f5-af73-815169340fbd service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:09.649720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 702/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:12.257081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64889c-b414-4667-98c9-328e37385238 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:12.257620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64889c-b414-4667-98c9-328e37385238 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:12.257736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64889c-b414-4667-98c9-328e37385238 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:12.259381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 690/1392] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:12.265011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f09-7b19-427b-b738-64ea31d83bcd None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:12.265475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f09-7b19-427b-b738-64ea31d83bcd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:12.265602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd8f09-7b19-427b-b738-64ea31d83bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:12.287535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-28bd8f09-7b19-427b-b738-64ea31d83bcd None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:12.289134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28bd8f09-7b19-427b-b738-64ea31d83bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:12.299492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 703/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:00:12.344938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64889c-b414-4667-98c9-328e37385238 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:12.360088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb64889c-b414-4667-98c9-328e37385238 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:12.365545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:12.366173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:12.366455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:12.366731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:12.376320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:12.377392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d8f4a-e2c2-4118-9164-aa681966e0b7 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:12.390852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 691/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:14.487209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304f22-0b93-4ff8-a87f-5b31ce31a3fa None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.487806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304f22-0b93-4ff8-a87f-5b31ce31a3fa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.487806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60304f22-0b93-4ff8-a87f-5b31ce31a3fa None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.489458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 704/1395] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:14.494485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5285105-b714-4ad3-9280-157ee2342793 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.494905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5285105-b714-4ad3-9280-157ee2342793 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.495177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5285105-b714-4ad3-9280-157ee2342793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.513832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d5285105-b714-4ad3-9280-157ee2342793 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:14.515021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d5285105-b714-4ad3-9280-157ee2342793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:14.522834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 692/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:14.566426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304f22-0b93-4ff8-a87f-5b31ce31a3fa None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:14.581075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60304f22-0b93-4ff8-a87f-5b31ce31a3fa None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:14.586463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:14.587095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.587381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.587662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.595778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:14.597386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069a304-5722-45b4-959a-88e1642cdbe6 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:14.606733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 705/1397] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:14.708136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072b1b-e3ce-465c-b790-004f9fb0b527 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.708857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072b1b-e3ce-465c-b790-004f9fb0b527 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.708857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50072b1b-e3ce-465c-b790-004f9fb0b527 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.710250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 693/1398] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:14.715901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877fd70-bd92-452d-84cc-6b73526076f4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.716231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877fd70-bd92-452d-84cc-6b73526076f4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.716507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877fd70-bd92-452d-84cc-6b73526076f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.735401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d877fd70-bd92-452d-84cc-6b73526076f4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:14.736689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d877fd70-bd92-452d-84cc-6b73526076f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:14.744713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 706/1399] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:14.802030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cab597-d722-418f-9eb3-ce8dd264dbcc None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.802321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cab597-d722-418f-9eb3-ce8dd264dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.802631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cab597-d722-418f-9eb3-ce8dd264dbcc None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.804223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 694/1400] 10.222.0.23 () {54 vars in 962 bytes} [Mon Jun 20 07:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:14.810360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed96c34-9aca-4e55-995f-3b46f5909489 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.810727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed96c34-9aca-4e55-995f-3b46f5909489 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.811089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed96c34-9aca-4e55-995f-3b46f5909489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.860494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fed96c34-9aca-4e55-995f-3b46f5909489 None None] MFA Rules not processed for user `e318b1b6386c4a2d9e7f324e8fc2cfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:14.861678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fed96c34-9aca-4e55-995f-3b46f5909489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:14.921098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 707/1401] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Jun 20 07:00:14 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:14.928674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cab597-d722-418f-9eb3-ce8dd264dbcc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:14.939768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cab597-d722-418f-9eb3-ce8dd264dbcc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:14.943295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:14.943958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:14.944277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:14.944559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:14.953700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:14.955129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebb6724-ac81-49d0-8b5a-f5db1e5cc8fc service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:14.964654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 695/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:15.679542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683575d-9d16-4ed7-b620-d777cb35039b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:15.680000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683575d-9d16-4ed7-b620-d777cb35039b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:15.680094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683575d-9d16-4ed7-b620-d777cb35039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:15.728130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d683575d-9d16-4ed7-b620-d777cb35039b None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:15.729326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d683575d-9d16-4ed7-b620-d777cb35039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:15.769873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:00:15.793875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 708/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:00:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:15.834298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910978-cd83-45c1-b095-738aed146e2f None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:15.834776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910978-cd83-45c1-b095-738aed146e2f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:15.834985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910978-cd83-45c1-b095-738aed146e2f None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:15.836549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 696/1404] 10.222.0.23 () {54 vars in 942 bytes} [Mon Jun 20 07:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:15.843016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9853a-5b51-4ec7-af22-82ce59ed0b35 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:15.843308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9853a-5b51-4ec7-af22-82ce59ed0b35 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:15.843619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9853a-5b51-4ec7-af22-82ce59ed0b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:15.893281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-65e9853a-5b51-4ec7-af22-82ce59ed0b35 None None] MFA Rules not processed for user `601afd44d1f54f939e87511ee463cc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:15.894486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-65e9853a-5b51-4ec7-af22-82ce59ed0b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:15.965017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 709/1405] 10.222.0.23 () {58 vars in 1039 bytes} [Mon Jun 20 07:00:15 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:15.974136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910978-cd83-45c1-b095-738aed146e2f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:15.987586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910978-cd83-45c1-b095-738aed146e2f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:15.992495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:15.993127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:15.993416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:15.993696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:16.002171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:16.003781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4228c2-c583-48e0-b007-7fa1bf0c8737 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:16.012742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 697/1406] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:00:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:00:16.693254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e77b-8a29-472a-bbeb-4158fdfff8ae None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:16.693934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e77b-8a29-472a-bbeb-4158fdfff8ae None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:16.693934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7e77b-8a29-472a-bbeb-4158fdfff8ae None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:16.698816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 710/1407] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:16.700550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396322a6-de1d-4837-b6b0-29b62138b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:16.700840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396322a6-de1d-4837-b6b0-29b62138b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:16.701119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396322a6-de1d-4837-b6b0-29b62138b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:16.710860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=67 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:00:16.729029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-396322a6-de1d-4837-b6b0-29b62138b0f6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:16.730228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-396322a6-de1d-4837-b6b0-29b62138b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:16.739429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 698/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:16.785481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e77b-8a29-472a-bbeb-4158fdfff8ae None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:16.800496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7e77b-8a29-472a-bbeb-4158fdfff8ae None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:16.806189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:16.807134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:16.807533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:16.807832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:16.816575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:16.818163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487036b-6759-4b9c-a258-0890c6a3addc service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:16.827756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 711/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:26.723664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:00:26.823270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=59 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:00:38.362628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878afda-a511-41c8-9a9c-81b4029fd70a None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:38.363196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878afda-a511-41c8-9a9c-81b4029fd70a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:38.363196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878afda-a511-41c8-9a9c-81b4029fd70a None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:38.364665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 699/1410] 10.222.0.23 () {54 vars in 918 bytes} [Mon Jun 20 07:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:38.369739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cd683-ebe2-4bd2-ac8a-28190de65e0e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:38.370161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cd683-ebe2-4bd2-ac8a-28190de65e0e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:38.370300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cd683-ebe2-4bd2-ac8a-28190de65e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:38.388448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d68cd683-ebe2-4bd2-ac8a-28190de65e0e None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:38.389648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d68cd683-ebe2-4bd2-ac8a-28190de65e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:38.398060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 712/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Jun 20 07:00:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:00:38.412342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe59dc-b853-41f3-9e38-788c62ecf252 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:38.412634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe59dc-b853-41f3-9e38-788c62ecf252 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:38.412918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe59dc-b853-41f3-9e38-788c62ecf252 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:38.414267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 700/1412] 10.222.0.23 () {54 vars in 918 bytes} [Mon Jun 20 07:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:38.419509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d1ddc-af8e-4535-b6d8-5d57042f8070 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:38.419796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d1ddc-af8e-4535-b6d8-5d57042f8070 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:38.420101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d1ddc-af8e-4535-b6d8-5d57042f8070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:38.437611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f42d1ddc-af8e-4535-b6d8-5d57042f8070 None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:38.438805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f42d1ddc-af8e-4535-b6d8-5d57042f8070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:38.447899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 713/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Mon Jun 20 07:00:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:00:40.943859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71c7e-2261-4e97-8cdc-1c8143b29e45 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:40.944409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71c7e-2261-4e97-8cdc-1c8143b29e45 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:40.944521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71c7e-2261-4e97-8cdc-1c8143b29e45 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:40.945857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:40.951083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646113a-0a6b-43b2-bcd8-dd7e67ede857 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:40.951515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646113a-0a6b-43b2-bcd8-dd7e67ede857 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:40.951618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646113a-0a6b-43b2-bcd8-dd7e67ede857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:40.969016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c646113a-0a6b-43b2-bcd8-dd7e67ede857 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:40.970123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c646113a-0a6b-43b2-bcd8-dd7e67ede857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:40.978213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:41.021507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71c7e-2261-4e97-8cdc-1c8143b29e45 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:41.036323 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71c7e-2261-4e97-8cdc-1c8143b29e45 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:41.041774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:41.042941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:41.043225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:41.043503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:41.051622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:41.053229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45656f52-02d4-4661-aafa-24d296bad31f service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:41.063192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 702/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:41.956638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684478ef-a948-43aa-91cd-d3b1139861ff None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:41.957143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684478ef-a948-43aa-91cd-d3b1139861ff None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:41.957239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684478ef-a948-43aa-91cd-d3b1139861ff None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:41.958618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:41.963342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb9dd-8361-4a1a-aa4e-ccc641527bad None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:41.963821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb9dd-8361-4a1a-aa4e-ccc641527bad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:41.964493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2eb9dd-8361-4a1a-aa4e-ccc641527bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:41.982097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3d2eb9dd-8361-4a1a-aa4e-ccc641527bad None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:41.983291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3d2eb9dd-8361-4a1a-aa4e-ccc641527bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:41.991126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:42.143302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684478ef-a948-43aa-91cd-d3b1139861ff None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:42.157235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684478ef-a948-43aa-91cd-d3b1139861ff None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:42.162387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:42.163009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:42.163291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:42.163565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:42.171315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:42.172908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f17786-534d-47c4-86fd-52732caa517c service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:42.182080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 716/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:43.072785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f4ad-40dd-48de-8bcc-af1824cca3ca None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:43.073361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f4ad-40dd-48de-8bcc-af1824cca3ca None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:43.073361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1f4ad-40dd-48de-8bcc-af1824cca3ca None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:43.074743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:43.079600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d80a-aec1-4a5d-b505-12673e1d3e8b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:43.079982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d80a-aec1-4a5d-b505-12673e1d3e8b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:43.080183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75d80a-aec1-4a5d-b505-12673e1d3e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:43.098077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6b75d80a-aec1-4a5d-b505-12673e1d3e8b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:43.099262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6b75d80a-aec1-4a5d-b505-12673e1d3e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:43.107533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:43.254335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1f4ad-40dd-48de-8bcc-af1824cca3ca None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:43.269413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1f4ad-40dd-48de-8bcc-af1824cca3ca None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:43.274785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:43.275422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:43.275707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:43.275989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:43.284744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:43.286322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca67de7-137b-4b2b-ae4b-4dd65bca550c service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:43.295687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 705/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:44.146927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23debfe-beac-4c87-9506-4a230d437b74 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:44.147487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23debfe-beac-4c87-9506-4a230d437b74 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:44.147487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23debfe-beac-4c87-9506-4a230d437b74 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:44.148890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:44.153884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb19b04-5b7f-493f-b843-78c7e4dd1606 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:44.154310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb19b04-5b7f-493f-b843-78c7e4dd1606 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:44.154466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb19b04-5b7f-493f-b843-78c7e4dd1606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:44.172428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9cb19b04-5b7f-493f-b843-78c7e4dd1606 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:44.173597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9cb19b04-5b7f-493f-b843-78c7e4dd1606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:44.181141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:44.329612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23debfe-beac-4c87-9506-4a230d437b74 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:44.343466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23debfe-beac-4c87-9506-4a230d437b74 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:44.348754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:44.349394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:44.349677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:44.349952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:44.358011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:44.359557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc72801-741c-4321-82a2-e867f735cd75 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:44.368664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 719/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:45.215991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a241-7082-40ec-aab1-77e22fe89d0d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:45.216595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a241-7082-40ec-aab1-77e22fe89d0d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:45.216595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77a241-7082-40ec-aab1-77e22fe89d0d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:45.217951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:45.222750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140e7e-5cab-4ea3-bef5-385ed0634726 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:45.223167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140e7e-5cab-4ea3-bef5-385ed0634726 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:45.223304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f140e7e-5cab-4ea3-bef5-385ed0634726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:45.240918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8f140e7e-5cab-4ea3-bef5-385ed0634726 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:45.242108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8f140e7e-5cab-4ea3-bef5-385ed0634726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:45.250294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:45.403870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77a241-7082-40ec-aab1-77e22fe89d0d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:45.418434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77a241-7082-40ec-aab1-77e22fe89d0d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:45.423739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:45.424401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:45.424685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:45.424960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:45.433606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:45.435214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bac6c-7472-40d8-b44d-b189be796919 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:45.444679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 708/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:46.258977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a26a-dbfb-4b91-9c82-214079328016 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:46.259508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a26a-dbfb-4b91-9c82-214079328016 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:46.259508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161a26a-dbfb-4b91-9c82-214079328016 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:46.261406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:46.266228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff8dc-55cb-41d9-9081-e74a62451fb5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:46.266619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff8dc-55cb-41d9-9081-e74a62451fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:46.266822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ff8dc-55cb-41d9-9081-e74a62451fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:46.285123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-959ff8dc-55cb-41d9-9081-e74a62451fb5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:46.286317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-959ff8dc-55cb-41d9-9081-e74a62451fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:46.294118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:46.442454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161a26a-dbfb-4b91-9c82-214079328016 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:46.457012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161a26a-dbfb-4b91-9c82-214079328016 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:46.462288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:46.462920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:46.463208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:46.463490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:46.471971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:46.473572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3020b-adfd-43fa-8d3d-79108e863256 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:46.483874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 722/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:47.282060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec874b0-7229-4021-827c-eca8bf3184f7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:47.282628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec874b0-7229-4021-827c-eca8bf3184f7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:47.282628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec874b0-7229-4021-827c-eca8bf3184f7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:47.283991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:47.288929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4c95b-b5a1-453b-aa15-618fea3d81f0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:47.289312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4c95b-b5a1-453b-aa15-618fea3d81f0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:47.289488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4c95b-b5a1-453b-aa15-618fea3d81f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:47.306907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f0e4c95b-b5a1-453b-aa15-618fea3d81f0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:47.308653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f0e4c95b-b5a1-453b-aa15-618fea3d81f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:47.316428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:00:47.468722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec874b0-7229-4021-827c-eca8bf3184f7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:47.483370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec874b0-7229-4021-827c-eca8bf3184f7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:47.488941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:47.489579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:47.489869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:47.490147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:47.499069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:47.500714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfcb1c-37f1-4357-9b88-f4f91c75042a service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:47.511204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 711/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:48.317625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107dfc-975b-4ec3-8efc-89245a6148bd None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:48.318239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107dfc-975b-4ec3-8efc-89245a6148bd None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:48.318348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50107dfc-975b-4ec3-8efc-89245a6148bd None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:48.319706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:48.325170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f720fc-4726-47b3-8e5e-edecf5315ba6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:48.325576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f720fc-4726-47b3-8e5e-edecf5315ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:48.325722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f720fc-4726-47b3-8e5e-edecf5315ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:48.344062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a3f720fc-4726-47b3-8e5e-edecf5315ba6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:48.345279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a3f720fc-4726-47b3-8e5e-edecf5315ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:48.353021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:48.504075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50107dfc-975b-4ec3-8efc-89245a6148bd None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:48.518616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50107dfc-975b-4ec3-8efc-89245a6148bd None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:48.523959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:48.524615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:48.524905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:48.525186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:48.533484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:48.535102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd0c53-b2be-49fb-9b6a-9fb7152c88e8 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:48.544764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 725/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:49.350587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731e66-bd16-49f4-bc0f-b5190818e518 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:49.351153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731e66-bd16-49f4-bc0f-b5190818e518 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:49.351153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731e66-bd16-49f4-bc0f-b5190818e518 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:49.352545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:49.357671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97bb-8097-4351-96b7-05bab7f1606d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:49.358108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97bb-8097-4351-96b7-05bab7f1606d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:49.358236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d97bb-8097-4351-96b7-05bab7f1606d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:49.375794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bd1d97bb-8097-4351-96b7-05bab7f1606d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:49.377027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd1d97bb-8097-4351-96b7-05bab7f1606d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:49.385212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:00:49.534713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4731e66-bd16-49f4-bc0f-b5190818e518 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:49.549039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4731e66-bd16-49f4-bc0f-b5190818e518 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:49.554551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:49.555183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:49.555490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:49.555766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:49.564377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:49.565939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5da4-429a-49f2-a6a8-26f9a7b9a566 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:49.575661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 714/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:51.503507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8225-1820-4962-b33b-1a9d1efd94c7 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:51.504086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8225-1820-4962-b33b-1a9d1efd94c7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:51.504086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8225-1820-4962-b33b-1a9d1efd94c7 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:51.505535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:51.511527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1cf36-9748-43fe-b92d-3933c014852c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:51.511949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1cf36-9748-43fe-b92d-3933c014852c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:51.512084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1cf36-9748-43fe-b92d-3933c014852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:51.530032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-79c1cf36-9748-43fe-b92d-3933c014852c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:51.531226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-79c1cf36-9748-43fe-b92d-3933c014852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:51.538986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:51.577116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae8225-1820-4962-b33b-1a9d1efd94c7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:51.587478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae8225-1820-4962-b33b-1a9d1efd94c7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:51.591522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:51.592168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:51.592454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:51.592726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:51.601677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:51.603082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb74edb-d8e8-483b-bd96-6e0699cface1 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:51.612291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 728/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:53.814107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad98a-c5ee-4084-a475-c9467f8939c0 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:53.814632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad98a-c5ee-4084-a475-c9467f8939c0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:53.814752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdad98a-c5ee-4084-a475-c9467f8939c0 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:53.816045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:53.821233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ddd78-2677-492e-bcb4-262aaebdc859 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:53.821630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ddd78-2677-492e-bcb4-262aaebdc859 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:53.821800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ddd78-2677-492e-bcb4-262aaebdc859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:53.840579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d43ddd78-2677-492e-bcb4-262aaebdc859 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:53.841772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d43ddd78-2677-492e-bcb4-262aaebdc859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:53.849624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:53.887445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdad98a-c5ee-4084-a475-c9467f8939c0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:53.898300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdad98a-c5ee-4084-a475-c9467f8939c0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:53.902373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:53.903004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:53.903289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:53.903567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:53.912401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:53.913823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96186d50-bf6a-40c0-8da2-0823b0930abd service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:53.923225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 717/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:56.266625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ec0b-ed0b-406e-829c-6f1788f71c6c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:56.267191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ec0b-ed0b-406e-829c-6f1788f71c6c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:56.267191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812ec0b-ed0b-406e-829c-6f1788f71c6c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:56.268612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:56.273693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129039f-6610-4427-89d8-94773ef67d11 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:56.274136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129039f-6610-4427-89d8-94773ef67d11 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:56.274248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129039f-6610-4427-89d8-94773ef67d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:56.292759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b129039f-6610-4427-89d8-94773ef67d11 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:56.293959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b129039f-6610-4427-89d8-94773ef67d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:56.301750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:56.339794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812ec0b-ed0b-406e-829c-6f1788f71c6c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:56.350281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812ec0b-ed0b-406e-829c-6f1788f71c6c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:56.353773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:56.354990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:56.355280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:56.355554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:56.364216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:56.365626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08acbd68-af58-4207-9bb3-f178b1dbfcc0 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:56.375010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 731/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:00:58.511982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1f89a-1600-4247-9c7e-07833a2fbace None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:58.512581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1f89a-1600-4247-9c7e-07833a2fbace None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:58.512581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1f89a-1600-4247-9c7e-07833a2fbace None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:58.513997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:00:58.519424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec408b6-f9ec-4b38-90a0-904334732b40 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:58.519823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec408b6-f9ec-4b38-90a0-904334732b40 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:58.519987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec408b6-f9ec-4b38-90a0-904334732b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:58.537745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2ec408b6-f9ec-4b38-90a0-904334732b40 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:00:58.539201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2ec408b6-f9ec-4b38-90a0-904334732b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:00:58.547000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:00:58.585602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1f89a-1600-4247-9c7e-07833a2fbace None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:00:58.596232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1f89a-1600-4247-9c7e-07833a2fbace None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:00:58.599713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:00:58.600385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:00:58.600673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:00:58.600951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:00:58.609836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:00:58.611274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35ad9-2db8-41db-95d2-9205567df2ad service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:00:58.621301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 720/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:00.717373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd361ea-260f-46b3-b763-a3248c6e78a9 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:00.717804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd361ea-260f-46b3-b763-a3248c6e78a9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:00.717938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd361ea-260f-46b3-b763-a3248c6e78a9 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:00.719355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 07:01:00.724494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1325-8fad-4809-865e-fd7bdcba7e99 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:00.724926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1325-8fad-4809-865e-fd7bdcba7e99 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:00.725043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b1325-8fad-4809-865e-fd7bdcba7e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:00.742301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-201b1325-8fad-4809-865e-fd7bdcba7e99 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:00.743524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-201b1325-8fad-4809-865e-fd7bdcba7e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:00.751171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:00.789315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd361ea-260f-46b3-b763-a3248c6e78a9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:00.799774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd361ea-260f-46b3-b763-a3248c6e78a9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:00.803235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:00.803859 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:00.804170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:00.804452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:00.813654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:00.815064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0d69b-15a1-45d2-bdbe-0aa72f715ebd service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:00.824907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 734/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:02.923065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e11b1a-79da-4192-81fe-0624f32f0e40 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:02.923621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e11b1a-79da-4192-81fe-0624f32f0e40 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:02.923621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e11b1a-79da-4192-81fe-0624f32f0e40 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:02.925015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:02.930133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60bf7-59e3-42c9-b670-cc59ab5d4567 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:02.930557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60bf7-59e3-42c9-b670-cc59ab5d4567 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:02.930694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60bf7-59e3-42c9-b670-cc59ab5d4567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:02.948946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-faa60bf7-59e3-42c9-b670-cc59ab5d4567 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:02.950133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-faa60bf7-59e3-42c9-b670-cc59ab5d4567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:02.957804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:02.995547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e11b1a-79da-4192-81fe-0624f32f0e40 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:03.005934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e11b1a-79da-4192-81fe-0624f32f0e40 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:03.009757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:03.010386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:03.010667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:03.010940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:03.019632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:03.021079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d8acb-9f85-4fde-966b-0ba1a242b406 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:03.030803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 723/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:05.091577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccfdc2-6798-4355-82f5-d5e169f195f1 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:05.092129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccfdc2-6798-4355-82f5-d5e169f195f1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:05.092129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccfdc2-6798-4355-82f5-d5e169f195f1 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:05.093517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:05.098809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e109-0631-4cdf-a043-ee079642e405 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:05.099233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e109-0631-4cdf-a043-ee079642e405 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:05.099376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0e109-0631-4cdf-a043-ee079642e405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:05.116668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-33a0e109-0631-4cdf-a043-ee079642e405 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:05.117839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-33a0e109-0631-4cdf-a043-ee079642e405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:05.125494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:05.164449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccfdc2-6798-4355-82f5-d5e169f195f1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:05.174751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccfdc2-6798-4355-82f5-d5e169f195f1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:05.178278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:05.178907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:05.179186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:05.179466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:05.188516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:05.189932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f9dbcc-e68e-456e-8bfe-98a2a948f09e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:05.199802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 737/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:07.295438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5f0a-8041-4038-8bd1-8ba608c5ab83 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:07.296002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5f0a-8041-4038-8bd1-8ba608c5ab83 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:07.296002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d5f0a-8041-4038-8bd1-8ba608c5ab83 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:07.297464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:07.302702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c83db-7a25-4331-97d6-8dfb23acc1e9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:07.303123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c83db-7a25-4331-97d6-8dfb23acc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:07.303264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c83db-7a25-4331-97d6-8dfb23acc1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:07.309227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=79 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:01:07.321593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d41c83db-7a25-4331-97d6-8dfb23acc1e9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:07.323276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d41c83db-7a25-4331-97d6-8dfb23acc1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:07.330954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:07.368201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d5f0a-8041-4038-8bd1-8ba608c5ab83 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:07.374418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=73 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:01:07.380975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d5f0a-8041-4038-8bd1-8ba608c5ab83 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:07.385410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:07.386044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:07.386330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:07.386605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:07.395467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:07.396938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79280195-f23a-4289-b17b-649ecd2bae17 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:07.408976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 726/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:09.451904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312861-8ab8-4d5c-8bf4-a1f2a7697a33 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:09.452490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312861-8ab8-4d5c-8bf4-a1f2a7697a33 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:09.452490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312861-8ab8-4d5c-8bf4-a1f2a7697a33 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:09.453891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:09.458937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720ea4-38ce-4013-a5e8-b40745b50496 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:09.459333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720ea4-38ce-4013-a5e8-b40745b50496 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:09.459494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7720ea4-38ce-4013-a5e8-b40745b50496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:09.477445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d7720ea4-38ce-4013-a5e8-b40745b50496 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:09.479157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d7720ea4-38ce-4013-a5e8-b40745b50496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:09.487099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:09.523166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65312861-8ab8-4d5c-8bf4-a1f2a7697a33 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:09.533793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65312861-8ab8-4d5c-8bf4-a1f2a7697a33 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:09.537360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:09.538011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:09.538292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:09.538563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:09.547750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:09.549230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132a332-b7a3-4946-b6cf-aa9c2345253d service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:09.558942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 740/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:11.661137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb35f4-11f2-438b-98f1-6a9ae89cb248 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:11.661668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb35f4-11f2-438b-98f1-6a9ae89cb248 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:11.661668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb35f4-11f2-438b-98f1-6a9ae89cb248 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:11.663069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:11.669848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a34ee-e3d6-4d88-b977-2311e0eb8d26 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:11.670279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a34ee-e3d6-4d88-b977-2311e0eb8d26 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:11.670406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a34ee-e3d6-4d88-b977-2311e0eb8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:11.688192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-590a34ee-e3d6-4d88-b977-2311e0eb8d26 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:11.689383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-590a34ee-e3d6-4d88-b977-2311e0eb8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:11.699032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:11.736286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb35f4-11f2-438b-98f1-6a9ae89cb248 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:11.747132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb35f4-11f2-438b-98f1-6a9ae89cb248 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:11.750893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:11.751531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:11.751818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:11.752100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:11.761858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:11.763280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d52d0c1-d425-4259-9b91-f72b255d4c18 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:11.772919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 729/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:21.682684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:01:21.768520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=14 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:01:31.704443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdf3a-ce2f-4733-a245-11f4d2793c61 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:31.705026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdf3a-ce2f-4733-a245-11f4d2793c61 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:31.705026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fdf3a-ce2f-4733-a245-11f4d2793c61 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:31.706411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 742/1471] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:31.712139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccef7e-faf9-4cfc-9555-b483cb0dc058 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:31.712532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccef7e-faf9-4cfc-9555-b483cb0dc058 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:31.712701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccef7e-faf9-4cfc-9555-b483cb0dc058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:31.730241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e6ccef7e-faf9-4cfc-9555-b483cb0dc058 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:31.731423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6ccef7e-faf9-4cfc-9555-b483cb0dc058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:31.739121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 730/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:31.787048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64364-2a6f-4374-a4cd-08ad28b85de3 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:31.787334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64364-2a6f-4374-a4cd-08ad28b85de3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:31.787610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c64364-2a6f-4374-a4cd-08ad28b85de3 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:31.789000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 743/1473] 10.222.0.23 () {54 vars in 962 bytes} [Mon Jun 20 07:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:31.793664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9766-7616-49ef-88b9-1c239bd8d15e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:31.793958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9766-7616-49ef-88b9-1c239bd8d15e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:31.794240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf9766-7616-49ef-88b9-1c239bd8d15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:31.812408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-17bf9766-7616-49ef-88b9-1c239bd8d15e None None] MFA Rules not processed for user `e318b1b6386c4a2d9e7f324e8fc2cfe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:31.813586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-17bf9766-7616-49ef-88b9-1c239bd8d15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:31.819928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 731/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Mon Jun 20 07:01:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:31.827515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c64364-2a6f-4374-a4cd-08ad28b85de3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:31.837829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c64364-2a6f-4374-a4cd-08ad28b85de3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:31.841237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:31.841847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:31.842154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:31.842427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:31.851367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:31.852842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b0ca9-1ec2-407d-b51d-635a6f1e33bd service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:31.862008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 744/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:33.984390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759bcd-69cd-4ab8-8f0a-1d5505082079 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:33.984901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759bcd-69cd-4ab8-8f0a-1d5505082079 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:33.984901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23759bcd-69cd-4ab8-8f0a-1d5505082079 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:33.986315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 732/1476] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:33.991318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f18-cc26-4bf3-ad23-9d6748f3d880 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:33.991721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f18-cc26-4bf3-ad23-9d6748f3d880 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:33.991880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e8f18-cc26-4bf3-ad23-9d6748f3d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:34.009836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9e4e8f18-cc26-4bf3-ad23-9d6748f3d880 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:34.011625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e4e8f18-cc26-4bf3-ad23-9d6748f3d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:34.019310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 745/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:34.041642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23759bcd-69cd-4ab8-8f0a-1d5505082079 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:34.054026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23759bcd-69cd-4ab8-8f0a-1d5505082079 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:34.058957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:34.059603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:34.059897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:34.060206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:34.060938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:34.062735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:01:34.062735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:01:34.062735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:01:34.063884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06d2257-a414-4a72-8e5e-938d12bc6ac5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:34.084045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 733/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Mon Jun 20 07:01:34 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:01:36.127378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571c0f-3825-4691-bcf7-482ea84daf89 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:36.127936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571c0f-3825-4691-bcf7-482ea84daf89 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:36.127936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d571c0f-3825-4691-bcf7-482ea84daf89 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:36.129342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 746/1479] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:36.134824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368c2c-9ff2-446c-9863-b90a4946d867 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:36.135257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368c2c-9ff2-446c-9863-b90a4946d867 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:36.135388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368c2c-9ff2-446c-9863-b90a4946d867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:36.153409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d0368c2c-9ff2-446c-9863-b90a4946d867 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:36.154597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d0368c2c-9ff2-446c-9863-b90a4946d867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:36.162288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 734/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:36.204427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d571c0f-3825-4691-bcf7-482ea84daf89 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:36.219228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d571c0f-3825-4691-bcf7-482ea84daf89 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:36.224601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:36.225239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:36.225531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:36.225810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:36.234404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:36.235981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00de51-e93a-4347-969e-82efa2913578 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:36.245336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 747/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:01:38.431522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f250-7349-4b18-a239-68e545c162f6 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:38.432083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f250-7349-4b18-a239-68e545c162f6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:38.432083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f250-7349-4b18-a239-68e545c162f6 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:38.433557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 735/1482] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:38.438825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ba52-d070-45d1-855c-a74a25e46d38 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:38.439224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ba52-d070-45d1-855c-a74a25e46d38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:38.439376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254ba52-d070-45d1-855c-a74a25e46d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:38.456444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0254ba52-d070-45d1-855c-a74a25e46d38 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:38.457611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0254ba52-d070-45d1-855c-a74a25e46d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:38.465746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 748/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:38.501684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f250-7349-4b18-a239-68e545c162f6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:38.512443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f250-7349-4b18-a239-68e545c162f6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:38.515872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:38.516525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:38.516810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:38.517087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:38.525799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:38.527725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10042f93-6f21-4dcc-8f42-9b286809ebb3 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:38.536958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 736/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:40.604218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ea4b-60d2-475d-9b65-34a09ede6918 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:40.604752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ea4b-60d2-475d-9b65-34a09ede6918 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:40.604873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09ea4b-60d2-475d-9b65-34a09ede6918 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:40.606161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 749/1485] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:40.611532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce013-41e9-41a4-b0e0-1ba5a1506d93 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:40.611944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce013-41e9-41a4-b0e0-1ba5a1506d93 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:40.612082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ce013-41e9-41a4-b0e0-1ba5a1506d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:40.629755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-774ce013-41e9-41a4-b0e0-1ba5a1506d93 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:40.630927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-774ce013-41e9-41a4-b0e0-1ba5a1506d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:40.639408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 737/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:40.677151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ea4b-60d2-475d-9b65-34a09ede6918 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:40.688138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09ea4b-60d2-475d-9b65-34a09ede6918 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:40.691604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:40.692274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:40.692561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:40.692836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:40.702332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:40.703756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90237c4-15d5-4813-9bc3-82e7f2c89ea7 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:40.713223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 750/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:42.883842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100198ed-3896-4f3d-b0f5-173fcaa7041d None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:42.884440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100198ed-3896-4f3d-b0f5-173fcaa7041d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:42.884440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100198ed-3896-4f3d-b0f5-173fcaa7041d None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:42.886063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 738/1488] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:42.891139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0267-747b-42d9-8ca0-42313a26bbfe None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:42.891535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0267-747b-42d9-8ca0-42313a26bbfe None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:42.891702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa0267-747b-42d9-8ca0-42313a26bbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:42.909408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c8aa0267-747b-42d9-8ca0-42313a26bbfe None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:42.910598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c8aa0267-747b-42d9-8ca0-42313a26bbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:42.918932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 751/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:42.959127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100198ed-3896-4f3d-b0f5-173fcaa7041d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:42.970373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100198ed-3896-4f3d-b0f5-173fcaa7041d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:42.973874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:42.974507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:42.974794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:42.975072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:42.984543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:42.986015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacc620-4842-4bbe-af80-ea96f0cd74e8 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:42.995300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 739/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:45.175689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c76f-eef4-40b2-b454-164efe58ba8b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:45.176286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c76f-eef4-40b2-b454-164efe58ba8b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:45.176286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763c76f-eef4-40b2-b454-164efe58ba8b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:45.177666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 752/1491] 10.222.0.23 () {54 vars in 926 bytes} [Mon Jun 20 07:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:01:45.183276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d444-cd74-4d33-bb49-93f0869d6164 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:45.183686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d444-cd74-4d33-bb49-93f0869d6164 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:45.183834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287d444-cd74-4d33-bb49-93f0869d6164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:45.201945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4287d444-cd74-4d33-bb49-93f0869d6164 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:01:45.203125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4287d444-cd74-4d33-bb49-93f0869d6164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:01:45.210811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 740/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Mon Jun 20 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:01:45.248655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763c76f-eef4-40b2-b454-164efe58ba8b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:01:45.259209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763c76f-eef4-40b2-b454-164efe58ba8b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:01:45.263740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:01:45.264401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:01:45.264687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:01:45.264967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:01:45.273700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:01:45.275118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbf01ed-73ca-4d08-bc1a-0141fecd2237 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:01:45.284585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 753/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:01:55.196315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=33 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:01:55.280359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:03:16.564612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.565200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.565200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.582763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:03:16.583956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:16.592792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 741/1494] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:03:16.601885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.602396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.602516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.620693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:03:16.621865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:16.629879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 754/1495] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:03:16.634537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.644460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea510df4-8c7f-4cc4-ae8d-a2fcaa9e7707 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.649009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.649655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.649944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.650226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.650950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:03:16.651260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.652329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.652329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:03:16.652329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:03:16.653215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.665016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 742/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:03:16.673228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.682261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8665b0a-2e0f-48b7-9cf9-d1fb848d0d20 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.686812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.687444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.687725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.687999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.688712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.690464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.690464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:03:16.690464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:03:16.691645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.727654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 755/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:03:16 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:03:16.732832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.742647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca37e61-5648-4e2c-95fa-5fb69e227925 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.798293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.798953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.799237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.799514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.800270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.801280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.801280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:03:16.801280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:03:16.802461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.821675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 743/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:03:16 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:03:16.825741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.835991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d04be-c488-4623-80e7-a4beffa53c3f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.840699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.841930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.842215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.842494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.843221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.844275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.844275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:03:16.844275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:03:16.845191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.858520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 756/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:03:16.862451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.872241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26814167-7ed5-477b-b9f8-7ffebecc9590 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.876790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.877430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.877711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.877986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.878694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.879697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.879697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:03:16.879697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:03:16.880624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1739fe-5f3a-4837-a7e6-d848761efc62 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.894265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 744/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:03:16.898199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:16.908503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b30896-39f0-402f-a0ae-de3cb09f5eee admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:16.912941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:16.913810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.914135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.914419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] PATH_INFO: `/v3/projects/f0ca310e8d974d048b8f32294aa9535b/users/9c62e2ecfc554b4389754bf22b4cfb82/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.928569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ca310e8d974d048b8f32294aa9535b, user_id=9c62e2ecfc554b4389754bf22b4cfb82, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:16.929601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:16.929601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:03:16.929601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:03:16.931513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9dbbf-0e13-43dc-a612-67ebcfaf6e68 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:16.943832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 757/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:03:16 2022] PUT /identity/v3/projects/f0ca310e8d974d048b8f32294aa9535b/users/9c62e2ecfc554b4389754bf22b4cfb82/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:03:16.955526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ec5c-df91-4c8e-afdd-d7e1f287e40a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:16.955829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ec5c-df91-4c8e-afdd-d7e1f287e40a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:16.956089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519ec5c-df91-4c8e-afdd-d7e1f287e40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:16.981620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d519ec5c-df91-4c8e-afdd-d7e1f287e40a None None] MFA Rules not processed for user `9c62e2ecfc554b4389754bf22b4cfb82`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:03:16.982803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d519ec5c-df91-4c8e-afdd-d7e1f287e40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:17.048088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 745/1502] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:03:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:03:17.061402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98516-4228-4cf5-8093-141d10437b87 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.061691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98516-4228-4cf5-8093-141d10437b87 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.061970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e98516-4228-4cf5-8093-141d10437b87 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.063359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 758/1503] 10.222.0.23 () {54 vars in 963 bytes} [Mon Jun 20 07:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:03:17.067956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa927-3a0b-498a-a868-f0f349857f05 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.068275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa927-3a0b-498a-a868-f0f349857f05 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.068556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9efa927-3a0b-498a-a868-f0f349857f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.087449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b9efa927-3a0b-498a-a868-f0f349857f05 None None] MFA Rules not processed for user `468b120f57fe4807b969c390cbcabfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:03:17.088588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b9efa927-3a0b-498a-a868-f0f349857f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:17.130146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:03:17.139442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 746/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Mon Jun 20 07:03:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:03:17.147202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e98516-4228-4cf5-8093-141d10437b87 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:17.158168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e98516-4228-4cf5-8093-141d10437b87 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:17.161682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:17.162308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.162594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.162874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.170973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:17.172411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:17.180980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 759/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:03:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:03:17.223010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c233-7f1e-413f-9947-8a7e1beeeb21 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.223304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c233-7f1e-413f-9947-8a7e1beeeb21 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.223590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401c233-7f1e-413f-9947-8a7e1beeeb21 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.224953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 747/1506] 10.222.0.23 () {52 vars in 829 bytes} [Mon Jun 20 07:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:03:17.228655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:17.289614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0f90c-9b06-4bf7-996c-476d01ae9a6d service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:17.295679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:17.296637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.296868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.297153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] PATH_INFO: `/v3/users/9c62e2ecfc554b4389754bf22b4cfb82` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.298353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c62e2ecfc554b4389754bf22b4cfb82)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:17.299378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:17.299378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:03:17.299378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:03:17.300607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:17.365467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c62e2ecfc554b4389754bf22b4cfb82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:17.377996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c62e2ecfc554b4389754bf22b4cfb82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:17.378758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c62e2ecfc554b4389754bf22b4cfb82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:17.388650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c62e2ecfc554b4389754bf22b4cfb82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:17.404590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:03:17.406756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c62e2ecfc554b4389754bf22b4cfb82'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:17.413555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 760/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:03:17 2022] DELETE /identity/v3/users/9c62e2ecfc554b4389754bf22b4cfb82 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:03:17.418164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.418453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.418733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.436312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:03:17.437501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:17.496479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 748/1508] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:03:17.508380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:17.510398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:17.570807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929213ea-75ad-48ce-8b44-56964d83e509 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:17.574327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:17.574975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.575277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.576183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.584972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:17.586406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f2e47a-6e47-4fb4-a50e-5e42326cb54f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:17.595729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 761/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:03:17.921134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:03:17.922668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:03:17.934826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a51f9-6778-480b-9a72-be0701f6aafe None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:03:17.940380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:03:17.941243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:03:17.941527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:03:17.941800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] PATH_INFO: `/v3/projects/f0ca310e8d974d048b8f32294aa9535b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:03:17.942994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ca310e8d974d048b8f32294aa9535b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:03:17.944024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:03:17.944024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:03:17.944024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:03:17.945242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:03:18.009777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ca310e8d974d048b8f32294aa9535b', 'request_id': 'req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:18.015969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ca310e8d974d048b8f32294aa9535b', 'request_id': 'req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:18.029342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ca310e8d974d048b8f32294aa9535b', 'request_id': 'req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:18.043137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] The token cache is being invalidate because project f0ca310e8d974d048b8f32294aa9535b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:03:18.043424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:18.044052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b46f875a-ca90-44dc-b5ba-dbd9d41b4bdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:03:18.046254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 749/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:03:17 2022] DELETE /identity/v3/projects/f0ca310e8d974d048b8f32294aa9535b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:03:27.592049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:03:28.040280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:04:54.330847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:54.331450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:54.331450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:54.332838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:04:54.337955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab267-1989-4a52-a16f-4623fc81aaa0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:54.338361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab267-1989-4a52-a16f-4623fc81aaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:54.338514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cab267-1989-4a52-a16f-4623fc81aaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:54.356323 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-82cab267-1989-4a52-a16f-4623fc81aaa0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:04:54.357510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82cab267-1989-4a52-a16f-4623fc81aaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:04:54.419810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:04:54.571714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:04:54.573434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:04:54.646165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7debe7-7e83-4913-9373-eff531cac194 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:04:54.651325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:04:54.651966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:54.652286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:54.652564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:54.660981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:04:54.662567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ae5af-84c0-4e20-8fcb-a8df6378e8c8 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:04:54.671740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 763/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:04:55.501474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ec10-6fbe-4de4-bf7e-2dde356038d7 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:55.502040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ec10-6fbe-4de4-bf7e-2dde356038d7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:55.502040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ec10-6fbe-4de4-bf7e-2dde356038d7 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:55.503429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:04:55.508414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81339ef-fbd3-42b3-ba67-da25d67b1832 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:55.508836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81339ef-fbd3-42b3-ba67-da25d67b1832 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:55.509073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81339ef-fbd3-42b3-ba67-da25d67b1832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:55.527172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f81339ef-fbd3-42b3-ba67-da25d67b1832 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:04:55.528404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f81339ef-fbd3-42b3-ba67-da25d67b1832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:04:55.537030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:04:55.688755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2ec10-6fbe-4de4-bf7e-2dde356038d7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:04:55.704444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2ec10-6fbe-4de4-bf7e-2dde356038d7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:04:55.709739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:04:55.710377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:55.710661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:55.710940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:55.719720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:04:55.721371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef9fda-250c-4870-b238-7f7b7bf4f5fc service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:04:55.730779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 752/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:04:56.325563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497550-5a5c-46d4-8973-d21661bb38d0 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:56.325853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497550-5a5c-46d4-8973-d21661bb38d0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:56.326128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a497550-5a5c-46d4-8973-d21661bb38d0 None None] PATH_INFO: `/v3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:56.327476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 765/1517] 10.222.0.48 () {54 vars in 925 bytes} [Mon Jun 20 07:04:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:04:56.333483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec0bb-26a4-4a6d-8177-b56d7edd7742 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:04:56.333785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec0bb-26a4-4a6d-8177-b56d7edd7742 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:04:56.334073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ec0bb-26a4-4a6d-8177-b56d7edd7742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:04:56.351774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e41ec0bb-26a4-4a6d-8177-b56d7edd7742 None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:04:56.353048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e41ec0bb-26a4-4a6d-8177-b56d7edd7742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:04:56.416948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 753/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jun 20 07:04:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:00.149667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076b9-d717-44be-bf9e-7302900139bc None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:00.150144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076b9-d717-44be-bf9e-7302900139bc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:00.150246 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076b9-d717-44be-bf9e-7302900139bc None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:00.151579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 766/1519] 10.222.0.23 () {54 vars in 960 bytes} [Mon Jun 20 07:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:00.157171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ca25-0a1c-485f-b872-baed2c355b6f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:00.157565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ca25-0a1c-485f-b872-baed2c355b6f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:00.157740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82ca25-0a1c-485f-b872-baed2c355b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:00.176187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5d82ca25-0a1c-485f-b872-baed2c355b6f None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:00.177384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5d82ca25-0a1c-485f-b872-baed2c355b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:00.185896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 754/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Mon Jun 20 07:05:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:00.193069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65076b9-d717-44be-bf9e-7302900139bc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:00.206419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65076b9-d717-44be-bf9e-7302900139bc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:00.211566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:00.212235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:00.212521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:00.212804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:00.222201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:00.223798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8941c79-203d-4ba5-a8b6-fa3a6ee1f573 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:00.233615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 767/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:05:06.954716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219f42-e955-4dff-a0a4-29957398784b None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:06.955286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219f42-e955-4dff-a0a4-29957398784b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:06.955286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219f42-e955-4dff-a0a4-29957398784b None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:06.956687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:06.961395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403b6d-9a08-4c93-ac25-c99eeb43529a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:06.961840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403b6d-9a08-4c93-ac25-c99eeb43529a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:06.961937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff403b6d-9a08-4c93-ac25-c99eeb43529a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:06.980331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ff403b6d-9a08-4c93-ac25-c99eeb43529a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:06.981559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ff403b6d-9a08-4c93-ac25-c99eeb43529a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:06.989884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:07.143447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65219f42-e955-4dff-a0a4-29957398784b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:07.157996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65219f42-e955-4dff-a0a4-29957398784b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:07.163233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:07.163856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:07.164169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:07.164451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:07.172422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:07.174002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1ee34e-658c-4285-9e4f-6e854c2d01d7 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:07.183133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 756/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:08.004375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c7ab-e17b-45be-b067-49be562396e4 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:08.004897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c7ab-e17b-45be-b067-49be562396e4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:08.004897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7c7ab-e17b-45be-b067-49be562396e4 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:08.006243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 07:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:08.010972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e96c2-82cc-4294-b521-5d29e9efdba2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:08.011348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e96c2-82cc-4294-b521-5d29e9efdba2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:08.011564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e96c2-82cc-4294-b521-5d29e9efdba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:08.030962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d51e96c2-82cc-4294-b521-5d29e9efdba2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:08.032169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d51e96c2-82cc-4294-b521-5d29e9efdba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:08.039812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 07:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:08.190360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7c7ab-e17b-45be-b067-49be562396e4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:08.204647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7c7ab-e17b-45be-b067-49be562396e4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:08.209949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:08.210583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:08.210871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:08.211153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:08.219800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:08.221425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab867d17-7722-4055-a2ae-9190fde8b28f service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:08.230855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 770/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:12.034619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:12.035142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:12.035260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:12.036603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:12.041995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4a2e-9831-4296-b884-be78255a0335 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:12.042437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4a2e-9831-4296-b884-be78255a0335 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:12.042544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4a2e-9831-4296-b884-be78255a0335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:12.060769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dbfe4a2e-9831-4296-b884-be78255a0335 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:12.061970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dbfe4a2e-9831-4296-b884-be78255a0335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:12.069771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:12.105559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:12.107230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:12.163802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f15bb6-9df8-4470-be73-021ebaef9235 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:12.167351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:12.167980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:12.168296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:12.168579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:12.177925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:12.179340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521742ca-7bae-4ef0-807f-d9b2f0521a89 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:12.184267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:05:12.189338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 759/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:14.225801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afb427-38c5-46c8-9d7c-e1192b7954f6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:14.226328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afb427-38c5-46c8-9d7c-e1192b7954f6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:14.226444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91afb427-38c5-46c8-9d7c-e1192b7954f6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:14.227750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:14.232838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b310c-eb04-480b-a351-f8825469f19c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:14.233265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b310c-eb04-480b-a351-f8825469f19c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:14.233387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b310c-eb04-480b-a351-f8825469f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:14.251961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7a9b310c-eb04-480b-a351-f8825469f19c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:14.253196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7a9b310c-eb04-480b-a351-f8825469f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:14.261054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:14.296287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afb427-38c5-46c8-9d7c-e1192b7954f6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:14.307749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91afb427-38c5-46c8-9d7c-e1192b7954f6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:14.311276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:14.311911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:14.312230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:14.312565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:14.321593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:14.323016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b314e-9457-4d59-b58b-39299d0ca922 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:14.332354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 773/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:17.735650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:17.736189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:17.736293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:17.737690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:17.742951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea298-05fb-4d4a-ba16-77c96200c8f7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:17.743348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea298-05fb-4d4a-ba16-77c96200c8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:17.743522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ea298-05fb-4d4a-ba16-77c96200c8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:17.749762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=62 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:05:17.762221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3c1ea298-05fb-4d4a-ba16-77c96200c8f7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:17.763394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3c1ea298-05fb-4d4a-ba16-77c96200c8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:17.771808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:18.075404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:18.077104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:18.133186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927aa33-8a62-414a-95f3-942a4ed18c59 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:18.136685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:18.137321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:18.137608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:18.138285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:18.148619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:18.150056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca403eba-4b6c-4d3e-ad51-eb09135b888f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:18.159292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 762/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:20.209114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:20.209702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:20.209702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:20.211083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 07:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:05:20.216281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde462-dc40-43c5-8ad8-e1304d9ca880 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:20.216733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde462-dc40-43c5-8ad8-e1304d9ca880 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:20.216844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bde462-dc40-43c5-8ad8-e1304d9ca880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:20.235968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-22bde462-dc40-43c5-8ad8-e1304d9ca880 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:05:20.237198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-22bde462-dc40-43c5-8ad8-e1304d9ca880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:20.245016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 07:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:05:20.546119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:05:20.547772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:05:20.560619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32115d1-4bc7-4577-9849-c047030fc0a6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:05:20.564223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:05:20.564894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:05:20.565182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:05:20.565463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:05:20.574388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:05:20.575801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcb70f-8e23-492a-b0b7-3203c84c6c30 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:05:20.585694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 776/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:05:30.230277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=32 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:05:30.581552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=8 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:46.537761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77982-2652-4a88-b061-0b8111f34e8b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.543009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77982-2652-4a88-b061-0b8111f34e8b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.543009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec77982-2652-4a88-b061-0b8111f34e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.543198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c22f2-42b4-4c57-a163-ef05ebaa0230 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.543198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c22f2-42b4-4c57-a163-ef05ebaa0230 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.543198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c22f2-42b4-4c57-a163-ef05ebaa0230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.580133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-243c22f2-42b4-4c57-a163-ef05ebaa0230 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.580133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-243c22f2-42b4-4c57-a163-ef05ebaa0230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.583243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3ec77982-2652-4a88-b061-0b8111f34e8b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.584806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3ec77982-2652-4a88-b061-0b8111f34e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.600555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 764/1540] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.604525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.605165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.605657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.607839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 777/1541] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.618858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.619215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.619435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.634708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.636338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.652732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 765/1542] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.658129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.659061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.660302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.674568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3205b208-1d81-49aa-89a3-0aeefd8d86c3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:46.683019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 778/1543] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.686521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:46.687537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.687822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.688296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.688797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.689761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:46.690278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:46.708134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dffc7-d1e2-47fb-a0a0-2248b6100b38 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:46.710363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c72e94-7f5e-4822-a125-80c88248941e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:46.726395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:46.726395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.726395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.726395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.726889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:46.726889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:46.726889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:46.726889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:06:46.726889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:06:46.727326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696463f3-0125-47f1-8bdd-29d471e1bcf6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:46.734699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 766/1544] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:06:46.736382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.736895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.737397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.755566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 779/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:06:46.757008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.757301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.757592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.771433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.773086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.784945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 767/1546] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.789296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.799146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.799146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.803731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4b01d-80b0-4187-af90-14fddcd5c26c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:46.808289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 780/1547] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.809002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:46.809424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.809972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.810484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.811724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.812679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:46.813926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:46.813926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:06:46.813926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:06:46.815553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f10313-8b01-4d7a-b9d2-3fef9446492f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:46.820877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e241109-9790-4117-8232-7d26182bcd7d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:46.825638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:46.826333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.827130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.827424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.828262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:46.830208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b28ce85-d156-4953-87d5-6c6c6d0d3be5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:46.846717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 768/1548] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/projects => generated 405 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:46.852585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d4c-5e0e-4f13-9ef4-9bcb7f103a80 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.853183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d4c-5e0e-4f13-9ef4-9bcb7f103a80 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.853694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad89d4c-5e0e-4f13-9ef4-9bcb7f103a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.860756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 781/1549] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:46.862190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300915f7-a2ad-4366-a54e-6b430148ccf8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.862685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300915f7-a2ad-4366-a54e-6b430148ccf8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.862970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300915f7-a2ad-4366-a54e-6b430148ccf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.875768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cad89d4c-5e0e-4f13-9ef4-9bcb7f103a80 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.877382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cad89d4c-5e0e-4f13-9ef4-9bcb7f103a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.883582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-300915f7-a2ad-4366-a54e-6b430148ccf8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.884876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-300915f7-a2ad-4366-a54e-6b430148ccf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.947823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 769/1550] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.948928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.949223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.949506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.959817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 782/1551] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.961462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.961756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.962036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.968854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.970213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.978405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 770/1552] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.979008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.980536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:46.981940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:46.988800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ea7f8-8be1-473a-b4a0-91297eec9dae None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:46.990229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 783/1553] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:46.990906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:46.996356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:46.997076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:46.997363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:46.997639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:46.998910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.000492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1d95d-a323-41cd-a7bf-400156d4ca70 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.000866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f917033-8efd-4626-ad4b-38d5725b5660 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.004903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.005534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.005813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.006083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.006757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.007754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:47.007754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:06:47.007754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:06:47.009082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbe804-48c7-419f-a6b9-7154930017d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.022461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 771/1554] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/users => generated 303 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.023979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eb523-5d08-47fa-9d73-32d4ced3195e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.024294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eb523-5d08-47fa-9d73-32d4ced3195e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.024737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5eb523-5d08-47fa-9d73-32d4ced3195e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.033721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 784/1555] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:46 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.033721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779491-1a1f-46b7-897f-d2606d5aa134 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.033721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779491-1a1f-46b7-897f-d2606d5aa134 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.033721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9779491-1a1f-46b7-897f-d2606d5aa134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.046674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-be5eb523-5d08-47fa-9d73-32d4ced3195e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.047816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-be5eb523-5d08-47fa-9d73-32d4ced3195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.050695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b9779491-1a1f-46b7-897f-d2606d5aa134 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.051902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b9779491-1a1f-46b7-897f-d2606d5aa134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.055640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 772/1556] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.056877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.057166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.057444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.065532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 785/1557] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.067156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.067448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.067730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.078290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.079568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.087504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 773/1558] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.088301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.088427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.089477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.098847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 786/1559] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.099521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa82e3-5fbb-4c07-9753-f50c0626e6ea None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.099648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.103981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.104746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.105013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.105292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.106020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.106326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.108488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.109557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9084fe2-c12c-4e17-9f38-31eb7ddbe3f3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.114106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.114744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.115039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.115314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.116037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.116382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.118004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.120795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 774/1560] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.121695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.131774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100dcd1-0488-42a5-84d9-dca74ab548cc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.140544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 787/1561] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.140544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.140917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.140917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.140917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.140917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.141551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.141551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15c0c-a016-4ac2-a429-68b9cb9ec3e4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.144892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bccf5a2-b5aa-4366-916c-9695631f275d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.149497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.150168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.153290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 775/1562] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.153449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.153449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.153449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.153678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.153678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.153678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.153909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1ef43-5606-4b47-8a52-f404c8571fb9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.166384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 788/1563] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.166384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.166384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.166384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.171198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.172417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.180485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 776/1564] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.181071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.183909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.185080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.190153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2204ace-c18f-46fa-bf2b-74d0b89179ee None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.192921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 789/1565] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.193797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.196799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.197440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.197720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.197998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.198723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.199029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.200649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.203059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7461f-d9d6-4efd-97c0-9a1372c83e66 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.207823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.208523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.208806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.209084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.209831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.210138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.211747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.212581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 777/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.213346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.230608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 790/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.230799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2f26c-fcac-487b-bb48-853e72f21893 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.230912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.235308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.235952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.236275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.236556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.236712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3d23a-33ad-4996-b4a7-a3f4f2d0dbdf admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.237246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.239195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.241320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.242049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.242344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.242623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.243413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.245327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.271879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 778/1568] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.271879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.274757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 791/1569] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.275445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.276221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:47.282415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:47.342411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31e4ea-b003-4ca2-be56-7fb90a6478e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.343768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a12ede-8319-456e-b82d-013a8a789eac admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.348309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.348977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.349270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.349546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.350273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.350736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.351429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.351718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.351894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.352006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.352909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.354502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.364449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 792/1570] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.365284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.368325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 779/1571] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.368325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.374735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e2aeb-2d53-4e02-8790-2cefb4c8f327 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.377582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1a84b-932b-4a90-9b88-423e077c826f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.379269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.379951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.380261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.380539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.381251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.381560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.382174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.382808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.383095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.383190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.383374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.384094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:47.384441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.386012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.395601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 793/1572] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.396202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.399210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 780/1573] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:47.400211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.406103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf48d0a-7a6e-4ba4-8a72-093ab8e8f32d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.411211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18512158-a372-431d-9ff4-8cf4f469bd8b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.411652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.412326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.412652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.412880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.413567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.415448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.415756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.416445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.416732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.417032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.417729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.419900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.446698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 794/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.447641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.453125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 781/1575] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.454516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.527464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42091316-de8f-4b03-992c-71ee30e2ecff admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.532593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.532748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4d1798-eba3-4efe-b31d-322692c3a0c6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.533328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.533538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.533810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.534493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.536456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.537594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.538258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.538806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.538806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.539490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.541476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.557144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 795/1576] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.558512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.564858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 782/1577] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.565749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.568040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9e38b-f0cb-44ac-9abc-0ea434d24b62 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.573198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.574064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.574351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.574633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] PATH_INFO: `/v3/projects/530da3341ec44dddbab5b0042706ac7b/users/1c618f9ab8bd47a3a9913078f8ae895e/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.575698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de471-bf26-4ea5-8490-f6b1b745b78a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.580747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.581613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.581900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.582202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] PATH_INFO: `/v3/projects/dc210e932f6d4f829e3f33df2413a526/users/45ccd23341ee46f39b379b90c2a83504/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.590056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] RBAC: Authorizing `identity:create_grant(project_id=530da3341ec44dddbab5b0042706ac7b, user_id=1c618f9ab8bd47a3a9913078f8ae895e, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.593449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.597260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc210e932f6d4f829e3f33df2413a526, user_id=45ccd23341ee46f39b379b90c2a83504, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.598403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:47.598403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:06:47.598403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:06:47.600486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.608475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 796/1578] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:47 2022] PUT /identity/v3/projects/530da3341ec44dddbab5b0042706ac7b/users/1c618f9ab8bd47a3a9913078f8ae895e/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:47.609314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.617090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 783/1579] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:47 2022] PUT /identity/v3/projects/dc210e932f6d4f829e3f33df2413a526/users/45ccd23341ee46f39b379b90c2a83504/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:47.617864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.640593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:47.658113 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:47.676516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4c577-81c6-4ffc-af9f-4f2a465e703b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.696317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9001b-6d8e-422c-a52b-650fd6f6ca80 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.701099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.701787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.702097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.702541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.703298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.705348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.733477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 784/1580] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.734379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.739486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.740193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.740482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.740791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.741491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.743541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.773165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 797/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.773165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.798670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13299c9a-64cb-4a89-94d4-8725e658f8d3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.831595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c55785-b6cb-40bb-9e22-73b935ab08dc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.836177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.836844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.837136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.837416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.838104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.840239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.854263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.855024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.855313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.855592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.856321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.858877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.860735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 798/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.861544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.871199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9d9b5-1721-4fb1-aca5-17a4b70737a6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.878087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.878753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.879040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.879319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.880057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.881679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e6485-b13c-4cf4-b78d-6e26002f80f6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.887273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 785/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.887273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.894579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 799/1584] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.895557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f3e5-710a-4ec0-a855-01fb9dfce248 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.899068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.899361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.899706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.900274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.900953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.901302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.901518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.902242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.903849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0b7a5-fc5a-4e65-9323-9a538b684737 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.914570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 786/1585] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:47.915773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.916065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.916371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.917415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.918889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.927341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 800/1586] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:47.928239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.939261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12e293-1038-40ca-8a89-3750543ed5c6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.942877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] MFA Rules not processed for user `45ccd23341ee46f39b379b90c2a83504`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:47.943950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.944083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:47.944651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.944936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.945218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.945898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.947811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:47.975720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 801/1587] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:47.976974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:47.986637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e556f7-6596-4ee5-b293-bfbb1a8bc9c8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:47.989402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:47.991840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:47.992521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:47.992805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:47.993081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:47.993765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:47.995818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.009880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:48.016727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 802/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/users => generated 284 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:48.017786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.028088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62697b20-88af-470a-9d3c-b7d8606ba362 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.028842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 787/1589] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:48.030198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.033829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.034483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.034769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.035046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.035766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.037386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.040667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112c0b8-de19-40c5-a1f3-cb64d2cd7de5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.046004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.046662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.046945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.047226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.047955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.051101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 803/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.051101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.051333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.058484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedb653-56cc-4023-a9ed-cdaaceda2bd9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.063212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 788/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.063212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.063424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.063886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.064199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.064484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.065195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.066765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.070549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6493746-9e44-4330-963c-b36719c1a2f7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.075243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.075887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.076233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.076496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.077211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.077583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 804/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.078379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.078806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.088953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 789/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.089603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.099685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d78b1-5352-4d9c-b712-fef379dc6456 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.104227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.104916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.105149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.105428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.106136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.108487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.119800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 790/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.121657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.131596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f191276-0950-4db8-b23a-4a29d96fe04d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.134070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd819c7-aef2-4134-af4a-12d748af38bd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.136272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.136902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.137189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.137476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.137708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.138179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.138350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.138638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.138950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.139796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.148381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.149810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.149967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 791/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.150716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.160496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 805/1596] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:48.160655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6de04-b54f-400e-90f8-16549caed23e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.161346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.165245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.165881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.166170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.166452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.167261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.169047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.170735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9532785e-1207-41f1-a627-4d53f855de98 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.175287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.175937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.176253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.176534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.177254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.178834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.192437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 806/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.192437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.197469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 792/1598] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:48.198746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.204585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746092ab-431b-4d56-b148-83ce64e903b3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.272745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db335fbd-d92d-40aa-adfa-02c41ca4342d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.275034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.275754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.275993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.276340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.277074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.278678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.279067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.280572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.280852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.281188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] PATH_INFO: `/v3/projects/31415828d04241249f14fb3ef6a3aa0a/users/f66183ca9d884f43befaace02099c7f2/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.291581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 807/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.292861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.298945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] RBAC: Authorizing `identity:create_grant(project_id=31415828d04241249f14fb3ef6a3aa0a, user_id=f66183ca9d884f43befaace02099c7f2, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.302379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.306621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34150b0c-3e12-4015-bda3-28f8330284b1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.312345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.313269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.313698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.314234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] PATH_INFO: `/v3/projects/f310d483bcb642e58d7f2f12b8e2fd15/users/ea022840c9a443249bd0c84ff9763e30/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.320983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 793/1600] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/31415828d04241249f14fb3ef6a3aa0a/users/f66183ca9d884f43befaace02099c7f2/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.320983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.340039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f310d483bcb642e58d7f2f12b8e2fd15, user_id=ea022840c9a443249bd0c84ff9763e30, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.343640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.366506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 808/1601] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/f310d483bcb642e58d7f2f12b8e2fd15/users/ea022840c9a443249bd0c84ff9763e30/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.367472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.407263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:48.418689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:48.424710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c856328-b96a-4bd3-975e-9f0094c1672f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.443572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a745bb-219f-4a81-994a-306e00e49727 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.450969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.452592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.453045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.453181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.453858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.455640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.467531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 809/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.471731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.483066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379b7a6-bfde-468e-9b43-53712afba9cd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.488884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.489876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.490115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.490610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] PATH_INFO: `/v3/projects/0e19a3f933284f088747a9d623e9ec24/users/785140aa617e46608e8b3cf89aac5a44/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.491869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.492585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.492872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.493151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.493886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.495495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.506127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 794/1603] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.506963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.517336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e19a3f933284f088747a9d623e9ec24, user_id=785140aa617e46608e8b3cf89aac5a44, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.519559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394f2c1-fb5e-404b-913e-f97e79e7070e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.520550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.524509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.525465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.525951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.526372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.527206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.530012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c105db-9391-4fa3-a1c4-6168a1a89f91 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.541156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 810/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/0e19a3f933284f088747a9d623e9ec24/users/785140aa617e46608e8b3cf89aac5a44/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.542424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.555307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 795/1605] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/users => generated 309 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:48.555307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1705f-41c6-4f7f-848d-c6d6573dff73 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.555307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1705f-41c6-4f7f-848d-c6d6573dff73 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.555307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1705f-41c6-4f7f-848d-c6d6573dff73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.574134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4de1705f-41c6-4f7f-848d-c6d6573dff73 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:48.575688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4de1705f-41c6-4f7f-848d-c6d6573dff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:48.611510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52870b-8bb5-4276-a67e-e7d0515da6f9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.617252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.618074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.618365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.618783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] PATH_INFO: `/v3/projects/b07b7ec64ab448338374bb16c29c7b1d/users/af0dee4e1dfb4f879dfe41622a9e23b1/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.636872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b07b7ec64ab448338374bb16c29c7b1d, user_id=af0dee4e1dfb4f879dfe41622a9e23b1, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.639935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.645281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 796/1606] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:48.646702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.647000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.647279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.666149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 811/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/b07b7ec64ab448338374bb16c29c7b1d/users/af0dee4e1dfb4f879dfe41622a9e23b1/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.666149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.677542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] MFA Rules not processed for user `ea022840c9a443249bd0c84ff9763e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:48.679035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:48.747185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa568c15-70f7-4928-b570-d199568dc6e5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.752281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.753193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.753479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.753760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] PATH_INFO: `/v3/projects/3185de36d1884f4aaba8492627fdf64a/users/fec5f5f520ae4dba9aba4e6a483ac33e/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.760304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 797/1608] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:48.761217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.769396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:48.770995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3185de36d1884f4aaba8492627fdf64a, user_id=fec5f5f520ae4dba9aba4e6a483ac33e, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.773540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a025d-8c18-4a16-9b0e-6254b9d01391 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.776476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d20a0b-9c5e-4b1d-a55d-e6f7f6644b34 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.785518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.786882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.787177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.787661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] PATH_INFO: `/v3/projects/96ee20ba4bcd469097e1a8815eee16e6/users/b974bab2e7934b57ab79128210c439a7/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.796672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 812/1609] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/3185de36d1884f4aaba8492627fdf64a/users/fec5f5f520ae4dba9aba4e6a483ac33e/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.798181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.798761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.799158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.809989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ee20ba4bcd469097e1a8815eee16e6, user_id=b974bab2e7934b57ab79128210c439a7, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.814220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.819875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:48.821464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:48.839078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 798/1610] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:48 2022] PUT /identity/v3/projects/96ee20ba4bcd469097e1a8815eee16e6/users/b974bab2e7934b57ab79128210c439a7/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:48.839078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.844903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:48.921648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 813/1611] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:48.922961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:48.932855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6553807-c7a2-4b7e-ad34-629905418aac admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:48.942196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:48.943264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.943675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.944494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.945445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:48.947410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f94b92-0ac5-44b5-8103-67a0780d8eb9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:48.962180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 799/1612] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:48.964587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a72e3-bf77-4755-920e-2558d04a27dc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:48.964949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a72e3-bf77-4755-920e-2558d04a27dc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:48.965335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a72e3-bf77-4755-920e-2558d04a27dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:48.990204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-960a72e3-bf77-4755-920e-2558d04a27dc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:48.991542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-960a72e3-bf77-4755-920e-2558d04a27dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.000805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 800/1613] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:49.002190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.002702 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.002993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.012928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be744b-2458-4f04-ae43-627a9b7798ed None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.038379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] MFA Rules not processed for user `fec5f5f520ae4dba9aba4e6a483ac33e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:49.039605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.087196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.088276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.088738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.089338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.104622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.106313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761bbe66-cfa8-4c69-b6e8-0869c864796c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.117327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 814/1614] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:49.118908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f10d-7a70-4aeb-8a0d-21de1f68e79b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.119395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f10d-7a70-4aeb-8a0d-21de1f68e79b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.119853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f3f10d-7a70-4aeb-8a0d-21de1f68e79b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.121274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 815/1615] 10.222.0.23 () {54 vars in 961 bytes} [Mon Jun 20 07:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:06:49.122315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.122654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.122948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.144886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 801/1616] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:49.145733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:49.146178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.147034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.147837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.157963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 816/1617] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:49.158693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.161876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1532c5-d341-4fa8-8d54-991cd4e5bfb4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.167671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.170579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.170879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.171166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.171886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede98223-5da0-49a2-89e4-3653ecc5d9ee None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.178177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.179083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.179743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.180049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] PATH_INFO: `/v3/projects/a336d0bffe9b48e8ac648091367d3019/users/e2b9f711aa73423c8623980b05137e3c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.187885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.189546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.203639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 802/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:49.204629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a336d0bffe9b48e8ac648091367d3019, user_id=e2b9f711aa73423c8623980b05137e3c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.208649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ede38d-8eea-46df-b31b-948dd3ca1668 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.222541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.238280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 817/1619] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:49 2022] PUT /identity/v3/projects/a336d0bffe9b48e8ac648091367d3019/users/e2b9f711aa73423c8623980b05137e3c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:49.242214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.244280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaeb5f3-ba1a-4f39-b184-b38d58b5ff66 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.245224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.245504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.286457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:49.289156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.326003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:49.340019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.341135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.343285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.343771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.400451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:49.461797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 818/1620] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:49.464287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.504202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.505741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.530423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 803/1621] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:49.531170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.547257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d751a66-98ca-42ba-b6bc-800633fcfa05 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.552370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.553514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.553899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.554189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.569009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.570549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.582651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 804/1622] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:49.583501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.583723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487b0db-02e3-4aed-af58-145fc3b9c74b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.589307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.590084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.590376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.590657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.593235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.594020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.594140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d144d-ef0b-4162-b94b-5637b013d111 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.600422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.601158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.601448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.601729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.602471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.604086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.615196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 805/1623] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:49.616187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.634528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57f10c-b31e-487a-8888-d184390545ca admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.637042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 819/1624] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:49.638079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.777410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.778337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.779217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.779809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.819770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491396c2-6ac1-4519-80d0-f792b7f45b60 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:49.833998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:49.837431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.837869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.838162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] PATH_INFO: `/v3/projects/a336d0bffe9b48e8ac648091367d3019/users/e2b9f711aa73423c8623980b05137e3c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.843923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a336d0bffe9b48e8ac648091367d3019, user_id=e2b9f711aa73423c8623980b05137e3c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.853219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.874181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:49.876085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33f0d8-a102-49dd-b97d-de3407ef2218 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:49.885614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 820/1625] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:49 2022] PUT /identity/v3/projects/a336d0bffe9b48e8ac648091367d3019/users/e2b9f711aa73423c8623980b05137e3c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:49.898743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:49.899671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 806/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:49.901496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:49.901897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:49.902187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:49.919067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:49.940344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] MFA Rules not processed for user `e2b9f711aa73423c8623980b05137e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:49.942652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:49.988155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:49.997104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859fc0-8803-48cf-8935-942d036d377b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.006119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.007907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.008487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.015583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.016919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.019363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.047533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 807/1627] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:50.057940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 821/1628] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:49 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:50.067411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.069791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.098552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbf3a6-44bc-4bb4-92f4-da6f6f490ac5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.112634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.113445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.115308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.116047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.118497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.122317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.155131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 808/1629] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:50.157256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.181726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993444b-85e9-47a2-9648-42cd5f7095db admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.192758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.193672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.193957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.194240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] PATH_INFO: `/v3/projects/7844683954bc4a0fae1aece060a88f26/users/d209f5e465b64bc981807799f2df93a1/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.211095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f3b7f-3a5c-452d-a5c1-d5cc12329970 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.219971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.221633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.221633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.221633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.228865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7844683954bc4a0fae1aece060a88f26, user_id=d209f5e465b64bc981807799f2df93a1, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.234606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe58d9b-4e73-4c91-b98c-779570c336d4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.245617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.247265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aab216-a087-4196-8d85-b708c88043ce service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.280426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 809/1630] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:50 2022] PUT /identity/v3/projects/7844683954bc4a0fae1aece060a88f26/users/d209f5e465b64bc981807799f2df93a1/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:50.282016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafff6-da2a-42e5-8a11-9a326124e19c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.282759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafff6-da2a-42e5-8a11-9a326124e19c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.282759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebafff6-da2a-42e5-8a11-9a326124e19c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.284791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 822/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:50 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:50.289354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 810/1632] 10.222.0.23 () {54 vars in 912 bytes} [Mon Jun 20 07:06:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:06:50.290524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebafff6-da2a-42e5-8a11-9a326124e19c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.298665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.299302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.299806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.355908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:50.364734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:50.452061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebafff6-da2a-42e5-8a11-9a326124e19c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.457957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.458663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.458939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.459226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.460357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.461648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.481285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 811/1633] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:50.483516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.526504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f853492-7f0d-48ae-9289-72947d357ba0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.545855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 823/1634] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:06:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:50.548698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.549779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.550189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.550591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] PATH_INFO: `/v3/projects/7844683954bc4a0fae1aece060a88f26/users/d209f5e465b64bc981807799f2df93a1/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.557062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7844683954bc4a0fae1aece060a88f26, user_id=d209f5e465b64bc981807799f2df93a1, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:50.567074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:50.567759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8acc579-92ee-4eda-9ea8-2e4ac7489a6f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:50.673156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 812/1635] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:50 2022] PUT /identity/v3/projects/7844683954bc4a0fae1aece060a88f26/users/d209f5e465b64bc981807799f2df93a1/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:50.673156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.673156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.673156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.724979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] MFA Rules not processed for user `d209f5e465b64bc981807799f2df93a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:50.729566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:50.763712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf8c51-dc68-46e8-abaf-9643c9ba33d5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:50.822474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:50.854872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:50.901966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:50.903490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:50.905648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:50.906320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:50.910143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 813/1636] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:50.913667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:51.026960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a5726-002c-4bc8-b0b7-1c1e948d9f8e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:51.033684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:51.034705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:51.035096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:51.035535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] PATH_INFO: `/v3/users/45ccd23341ee46f39b379b90c2a83504` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:51.039757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ccd23341ee46f39b379b90c2a83504)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:51.042020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: WARNING py.warnings [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:51.042020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: warnings.warn(msg) Jun 20 07:06:51.042020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: Jun 20 07:06:51.047006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:51.049081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:51.050860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:51.071385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 824/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:51.072715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:51.094300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afabb7-931d-492e-be99-8ce1b259f1b4 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:51.101533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:51.102518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:51.103010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:51.103732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:51.149578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:51.151878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99558ac8-34f9-4fdb-9a5e-110fd4772ddf service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:51.181356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ccd23341ee46f39b379b90c2a83504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:51.185345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ccd23341ee46f39b379b90c2a83504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:51.186063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ccd23341ee46f39b379b90c2a83504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:51.191845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 825/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:51.201938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ccd23341ee46f39b379b90c2a83504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:51.237286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ccd23341ee46f39b379b90c2a83504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:51.290552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 814/1639] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:06:50 2022] DELETE /identity/v3/users/45ccd23341ee46f39b379b90c2a83504 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:51.295607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cad-e4ab-4254-b99f-2d70d56800a7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:51.295908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cad-e4ab-4254-b99f-2d70d56800a7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:51.296219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b4cad-e4ab-4254-b99f-2d70d56800a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:51.336421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-425b4cad-e4ab-4254-b99f-2d70d56800a7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:51.339200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-425b4cad-e4ab-4254-b99f-2d70d56800a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:51.515465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 826/1640] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:51.546693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:51.549183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:51.650415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:51.677714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2e551-1e5b-4061-84ef-dfe884684dd9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:51.683564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:51.684578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:51.685037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:51.685407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:51.698681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:51.700567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c584ee6-ce2b-47ba-a4ae-0d6569740460 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:51.714675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 815/1641] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:52.497602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70112bf1-13ec-46f4-a96e-ee1ad08bcbaa None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:52.498549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70112bf1-13ec-46f4-a96e-ee1ad08bcbaa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:52.499123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70112bf1-13ec-46f4-a96e-ee1ad08bcbaa None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:52.501033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 827/1642] 10.222.0.23 () {54 vars in 961 bytes} [Mon Jun 20 07:06:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:06:52.510782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ff1-c2e8-4741-8933-2109b5dce425 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:52.511017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ff1-c2e8-4741-8933-2109b5dce425 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:52.511485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52ff1-c2e8-4741-8933-2109b5dce425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:52.536447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c0c52ff1-c2e8-4741-8933-2109b5dce425 None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:52.537997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c0c52ff1-c2e8-4741-8933-2109b5dce425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:52.651611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:52.651920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 816/1643] 10.222.0.23 () {58 vars in 1058 bytes} [Mon Jun 20 07:06:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:52.652402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:52.652846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:52.654957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 828/1644] 10.222.0.23 () {54 vars in 912 bytes} [Mon Jun 20 07:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:06:52.661002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:52.661258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:52.661493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:52.705521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:52.709271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:52.741423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 817/1645] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:06:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:52.756204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:52.759453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:52.915134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe4745-a488-45d3-bc22-33994e507874 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:52.926180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:52.926882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:52.927114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:52.927440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:52.946609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:52.948697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:53.018700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 829/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:06:53.232372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:53.234355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:53.262958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62661afd-a222-469d-a9c6-5cbd76c13b2f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:53.277119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:53.278043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:53.278331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:53.278611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] PATH_INFO: `/v3/projects/dc210e932f6d4f829e3f33df2413a526` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:53.283864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc210e932f6d4f829e3f33df2413a526)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:53.287799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:53.449609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc210e932f6d4f829e3f33df2413a526', 'request_id': 'req-5fe6390c-0153-41be-a59c-4496c68ca647'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.471083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc210e932f6d4f829e3f33df2413a526', 'request_id': 'req-5fe6390c-0153-41be-a59c-4496c68ca647'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.490128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc210e932f6d4f829e3f33df2413a526', 'request_id': 'req-5fe6390c-0153-41be-a59c-4496c68ca647'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.524166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] The token cache is being invalidate because project dc210e932f6d4f829e3f33df2413a526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:06:53.524166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.526237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.536400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 818/1647] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:06:53 2022] DELETE /identity/v3/projects/dc210e932f6d4f829e3f33df2413a526 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:53.537614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:53.558306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b624c-8ec0-4aa6-a38e-747bf2dd4061 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:53.565020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:53.726368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:53.728809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:53.729237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:53.730260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] PATH_INFO: `/v3/users/e2b9f711aa73423c8623980b05137e3c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:53.733444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b9f711aa73423c8623980b05137e3c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:53.741531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:53.853319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b9f711aa73423c8623980b05137e3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.863932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b9f711aa73423c8623980b05137e3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.865228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b9f711aa73423c8623980b05137e3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.879741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b9f711aa73423c8623980b05137e3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.898823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-44987dbb-c1fe-4a5a-932b-de8c7f9b3f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b9f711aa73423c8623980b05137e3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:53.910968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 830/1648] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:06:53 2022] DELETE /identity/v3/users/e2b9f711aa73423c8623980b05137e3c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:54.024571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:54.026053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:54.046305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6390c-0153-41be-a59c-4496c68ca647 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:54.134306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:54.151699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:54.153155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:54.153556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:54.154909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] PATH_INFO: `/v3/users/fec5f5f520ae4dba9aba4e6a483ac33e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:54.158175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec5f5f520ae4dba9aba4e6a483ac33e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:54.160797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:54.278373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec5f5f520ae4dba9aba4e6a483ac33e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:54.280576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec5f5f520ae4dba9aba4e6a483ac33e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:54.284566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec5f5f520ae4dba9aba4e6a483ac33e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:54.297644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec5f5f520ae4dba9aba4e6a483ac33e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:54.314730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec5f5f520ae4dba9aba4e6a483ac33e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:54.333166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 819/1649] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:06:54 2022] DELETE /identity/v3/users/fec5f5f520ae4dba9aba4e6a483ac33e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:54.338034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:54.338337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:54.339030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:54.396766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:54.398350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:54.506537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 831/1650] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:54.528424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:54.532714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:54.722655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b9021-7cd8-4a77-a5bd-316759680865 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:54.731826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:54.732578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:54.733363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:54.733646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:54.754835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:54.755572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f735-4757-4277-ae78-f0adec522f80 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:54.784195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 820/1651] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:54.885091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:54.887966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:54.912325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa6000-3904-4ca5-9d18-d234bdcb50e4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:54.920465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:54.922933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:54.923401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:54.923986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] PATH_INFO: `/v3/users/ea022840c9a443249bd0c84ff9763e30` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:54.925791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea022840c9a443249bd0c84ff9763e30)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:54.928024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:54.977521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:55.025101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea022840c9a443249bd0c84ff9763e30'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.035870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea022840c9a443249bd0c84ff9763e30'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.036759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea022840c9a443249bd0c84ff9763e30'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.049502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea022840c9a443249bd0c84ff9763e30'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.072576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea022840c9a443249bd0c84ff9763e30'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.081682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 832/1652] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:06:54 2022] DELETE /identity/v3/users/ea022840c9a443249bd0c84ff9763e30 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:55.086144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.086638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.086943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.112243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:55.113444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:55.194904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 821/1653] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:55.217015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:55.219074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:55.321660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:55.323316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:55.336537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdff3e2-aee3-4d2f-aad2-0c4803fd9e53 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:55.356356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:55.357247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.359075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44db8c-d4fb-4e30-b643-b9a22f34ed67 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:55.359979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.360588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.372655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:55.375707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.376095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.376707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] PATH_INFO: `/v3/projects/a336d0bffe9b48e8ac648091367d3019` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.379857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a336d0bffe9b48e8ac648091367d3019)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:55.393354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:55.396999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:55.400995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8608e0-306e-4844-9449-fe13dd4c9177 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:55.431858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:55.432688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 833/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:55.489212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a336d0bffe9b48e8ac648091367d3019', 'request_id': 'req-8afe2825-dbaf-4fa0-8091-7e93b84c0736'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.499029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a336d0bffe9b48e8ac648091367d3019', 'request_id': 'req-8afe2825-dbaf-4fa0-8091-7e93b84c0736'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.508700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a336d0bffe9b48e8ac648091367d3019', 'request_id': 'req-8afe2825-dbaf-4fa0-8091-7e93b84c0736'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.532682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] The token cache is being invalidate because project a336d0bffe9b48e8ac648091367d3019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:06:55.533285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.535735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8afe2825-dbaf-4fa0-8091-7e93b84c0736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:55.537687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 822/1655] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:06:55 2022] DELETE /identity/v3/projects/a336d0bffe9b48e8ac648091367d3019 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:55.543071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.544208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.544710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.575040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:55.576819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:55.755400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 834/1656] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:06:55.759869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.760879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.762261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.821111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:55.822943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:55.832556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 823/1657] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:55.839270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:55.856221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d93dd9-3221-4609-92ef-1e5bb6981aea None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:55.865932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:55.867505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.868154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.868733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.870251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:55.870805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:55.873164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:55.890861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 835/1658] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:55.895574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:55.916244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c042844-3089-4c5e-b3e3-24818735ee34 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:55.927378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:55.928623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:55.930193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:55.930730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:55.931930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:55.938482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:55.989255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 824/1659] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:55 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:06:55.995973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.013353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0d09-a6af-4fb4-91a5-ff778d28b480 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.062594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:56.121640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.123722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.124457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.124836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.125781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.128982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.168884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 836/1660] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:55 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:56.171336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.190710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb21881-bc77-41f3-bb99-8b8fdbc90695 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.215951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.218854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.220376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.220985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.222671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.225011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.247508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 825/1661] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:56.259955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.271805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2bf19b-d401-4e17-be59-28f9210a8d0f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.280940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.287815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.288460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.298828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.299688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.301890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a44d6-4e2b-46f3-ade1-99c0eb84a165 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.322599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 837/1662] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:56.327526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.345658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aafe6-35d3-44ed-9f53-e41cd5d6a18f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.364628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.366788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.368099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.378765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] PATH_INFO: `/v3/projects/3b660d743d01487fa2e57f14b631de75/users/c3050a754d2e4ddc8d43aa397873e827/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.430965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b660d743d01487fa2e57f14b631de75, user_id=c3050a754d2e4ddc8d43aa397873e827, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.443266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.504167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 826/1663] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:56 2022] PUT /identity/v3/projects/3b660d743d01487fa2e57f14b631de75/users/c3050a754d2e4ddc8d43aa397873e827/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:56.521878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.524298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.524670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.524960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.525877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:56.612745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] MFA Rules not processed for user `c3050a754d2e4ddc8d43aa397873e827`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:56.614167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:56.735034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2524b-f9f0-460e-98c7-723f7318b59c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.748640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.750743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.752033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.753926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] PATH_INFO: `/v3/projects/f310d483bcb642e58d7f2f12b8e2fd15` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.759799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f310d483bcb642e58d7f2f12b8e2fd15)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.764074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.774362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 838/1664] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:56.775796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.808075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bd3c-6d7d-4bfd-a8ea-72ee0fded2a0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:56.815150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:56.816488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.816915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.818987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:56.819868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:56.829388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:56.879742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 839/1665] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:56 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:56.881787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:56.889418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:56.908478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f310d483bcb642e58d7f2f12b8e2fd15', 'request_id': 'req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:56.927165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f310d483bcb642e58d7f2f12b8e2fd15', 'request_id': 'req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:56.943008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f310d483bcb642e58d7f2f12b8e2fd15', 'request_id': 'req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:56.970542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] The token cache is being invalidate because project f310d483bcb642e58d7f2f12b8e2fd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:06:56.970938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:56.973070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fcffd9ba-cb18-4497-8a04-928f47f4bd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:56.976610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 827/1666] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:06:56 2022] DELETE /identity/v3/projects/f310d483bcb642e58d7f2f12b8e2fd15 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:06:56.981644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21dcba-45c6-4c46-ab4f-b3821ac49f53 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:56.982075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21dcba-45c6-4c46-ab4f-b3821ac49f53 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:56.982512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21dcba-45c6-4c46-ab4f-b3821ac49f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.019552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595a663-bfb4-4b54-85ae-eacd65d3f334 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.026937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.027793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.028084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.028546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.029293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.031504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.035475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fb21dcba-45c6-4c46-ab4f-b3821ac49f53 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:57.040052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fb21dcba-45c6-4c46-ab4f-b3821ac49f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.058751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 840/1667] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:56 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:57.063646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.065151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 828/1668] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:06:57.073250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.073378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.073849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.074288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.091242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc193720-7808-436a-bb14-74446c59c215 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.098526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.098792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.099136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.099414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.100497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.102722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.104626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:57.118160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:57.121427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.122193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 841/1669] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:57.124558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.136790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cbd51-3fc4-42a9-89ce-ec6586ac9f12 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.145173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.145600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 829/1670] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:57.149976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.154094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.154464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.154822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] PATH_INFO: `/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292/users/b9372cd583db4107903c4f95f713690b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.176534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae19fc7-d86d-4e7b-ab16-f26bed0c8f70 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.185337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a6dcaad3a419bb863fb263d7f8292, user_id=b9372cd583db4107903c4f95f713690b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.188798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.195873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.196742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.197089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.197430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.198414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:06:57.198925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.201078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.225061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 842/1671] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:57 2022] PUT /identity/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292/users/b9372cd583db4107903c4f95f713690b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:57.229942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.252891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 830/1672] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:06:57.259797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.295635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc4ab2-62e3-4416-8138-329b389f5e93 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.404068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382704aa-8612-4114-b569-28ad09a93ecb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.418870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.419558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.420409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.420687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.423599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.425274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.441811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.442502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.442954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.443246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.443966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.447726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 843/1673] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:57.448697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.450465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.456661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.464060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c0967-623e-4aed-a71c-77b25aa2e784 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.469156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.470219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.470326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.470573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] PATH_INFO: `/v3/projects/3185de36d1884f4aaba8492627fdf64a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.479656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3185de36d1884f4aaba8492627fdf64a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.480827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: WARNING py.warnings [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 07:06:57.480827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: warnings.warn(msg) Jun 20 07:06:57.480827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: Jun 20 07:06:57.483358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.547232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 831/1674] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:06:57 2022] POST /identity/v3/projects => generated 363 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:57.548124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.591744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3185de36d1884f4aaba8492627fdf64a', 'request_id': 'req-389c1233-40b6-444f-8b11-2ff91a0dc375'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:57.608665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3185de36d1884f4aaba8492627fdf64a', 'request_id': 'req-389c1233-40b6-444f-8b11-2ff91a0dc375'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:57.622716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3185de36d1884f4aaba8492627fdf64a', 'request_id': 'req-389c1233-40b6-444f-8b11-2ff91a0dc375'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:57.663765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] The token cache is being invalidate because project 3185de36d1884f4aaba8492627fdf64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:06:57.664336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:57.665208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:57.667293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 844/1675] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:06:57 2022] DELETE /identity/v3/projects/3185de36d1884f4aaba8492627fdf64a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:57.668331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.671712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.710497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55adf8-e746-4379-a51d-9ba336322f8e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.729156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.730207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.730594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.731123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] PATH_INFO: `/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292/users/b9372cd583db4107903c4f95f713690b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.735957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a6dcaad3a419bb863fb263d7f8292, user_id=b9372cd583db4107903c4f95f713690b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.738794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.778528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 832/1676] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:57 2022] PUT /identity/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292/users/b9372cd583db4107903c4f95f713690b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:57.778528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:57.778528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:57.791511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:57.819085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c1233-40b6-444f-8b11-2ff91a0dc375 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.823647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:57.923654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69486fd-c5c1-4087-9425-fec597d437b3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:57.924056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.925676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.926093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.926547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.929557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.930040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:57.930996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.931291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.931624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] PATH_INFO: `/v3/users/d209f5e465b64bc981807799f2df93a1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:57.932968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d209f5e465b64bc981807799f2df93a1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:57.933490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ce3d85-1901-4548-9d8e-d6af72469677 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.935224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:57.980512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 845/1677] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:06:57 2022] POST /identity/v3/users => generated 282 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:06:57.984633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:57.985192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:57.986971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.026367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd209f5e465b64bc981807799f2df93a1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:58.027087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd209f5e465b64bc981807799f2df93a1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:58.027731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd209f5e465b64bc981807799f2df93a1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:58.032352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] MFA Rules not processed for user `b9372cd583db4107903c4f95f713690b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:58.034569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.047320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd209f5e465b64bc981807799f2df93a1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:58.074720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd209f5e465b64bc981807799f2df93a1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:06:58.105377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 833/1678] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:06:57 2022] DELETE /identity/v3/users/d209f5e465b64bc981807799f2df93a1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:06:58.109158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.112787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.120093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 846/1679] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:58.135536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.137582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.259622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20e66b-30f2-4848-b7f4-f700b6f22cf2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.262447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5e958-cc86-4bee-9ec8-65e2e2ef43a7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.266982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.267824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.268198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.268426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.269176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.275780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:58.299796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 834/1680] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:58.301970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:58.303530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.330383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ca297-185c-422d-a172-640e4df968a2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.335599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.336482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.336857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.337147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.337724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.337885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.338601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.338985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.339487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.339650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:58.351794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 835/1681] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:06:58.355554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.368228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9a320-16e2-4a9b-a6c5-703d6867b312 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.374286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.375409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.375708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.376745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] PATH_INFO: `/v3/projects/af55321b0ab5484399028d8cf04ae1bf/users/6cf35ec82546401f9bb339741ea6adee/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.396161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af55321b0ab5484399028d8cf04ae1bf, user_id=6cf35ec82546401f9bb339741ea6adee, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.399149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2833ac1-4979-4e70-bdd4-ebcff33969c2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:58.411453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:06:58.417864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 836/1682] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:06:58 2022] PUT /identity/v3/projects/af55321b0ab5484399028d8cf04ae1bf/users/6cf35ec82546401f9bb339741ea6adee/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:06:58.429159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.429564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.429924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.436820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.438995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:58.457559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 847/1683] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:06:58.475924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] MFA Rules not processed for user `6cf35ec82546401f9bb339741ea6adee`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:06:58.477572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.605809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 837/1684] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:06:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:06:58.619430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.632178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20a498-9405-4699-954c-12b4d20a15dd service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.634159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:06:58.635725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.776929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2269000-3417-4738-9a21-a89ed455e5bd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:06:58.782197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.782860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.783148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.783427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.803724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:06:58.804594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:06:58.804886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:06:58.805166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:06:58.816873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:06:58.827677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.838843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:58.888299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 848/1685] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:58 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:06:58.985397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:06:58.987497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb29668-4690-432b-b1fd-acac5361c0f7 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:06:59.020830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 838/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:06:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:02.461452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:02.463142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:02.649918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b629674-798c-4bc3-b88c-8ee9db11a660 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:02.658849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:02.659823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:02.676048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:02.676488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] PATH_INFO: `/v3/projects/7844683954bc4a0fae1aece060a88f26` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:02.680029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7844683954bc4a0fae1aece060a88f26)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:02.682504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:02.823277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:02.907675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7844683954bc4a0fae1aece060a88f26', 'request_id': 'req-373df492-a3c9-4419-b43a-3032e4394f32'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:02.942823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7844683954bc4a0fae1aece060a88f26', 'request_id': 'req-373df492-a3c9-4419-b43a-3032e4394f32'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:02.960353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7844683954bc4a0fae1aece060a88f26', 'request_id': 'req-373df492-a3c9-4419-b43a-3032e4394f32'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:02.994842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] The token cache is being invalidate because project 7844683954bc4a0fae1aece060a88f26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:02.994842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:03.013224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-373df492-a3c9-4419-b43a-3032e4394f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:03.026132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 849/1687] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:02 2022] DELETE /identity/v3/projects/7844683954bc4a0fae1aece060a88f26 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:03.030868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.031380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.031544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.085304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:03.102871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:03.191567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:03.251490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 839/1688] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:03.259193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.259634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.259927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.317621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:03.318785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:03.360255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 850/1689] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:03.361184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:03.381025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876cf78-98a4-4dd4-a47e-7a70d363c558 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:03.395666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:03.396561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.396947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.397388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.398180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:03.398657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:03.406419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:03.440057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 840/1690] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:03.445079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:03.469584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b83d9-783f-49c4-ac84-351e57d48527 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:03.482909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:03.485104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.485410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.485955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.488055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:03.490835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f33e4-35bd-4cd4-b121-0565d028e91e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:03.580455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:03.580637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 851/1691] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:03 2022] POST /identity/v3/projects => generated 407 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:03.595288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef90ef-368d-4344-a105-7a03f1284fab admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:03.766590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.767002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.767494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.848993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] MFA Rules not processed for user `f66183ca9d884f43befaace02099c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:03.857842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:03.859606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:03.863792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.864309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.864598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.867794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:03.869826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:03.931233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 841/1692] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:03 2022] POST /identity/v3/users => generated 304 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:03.932244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:03.954184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f334e-9bbe-4a21-b74c-891392e7ae9a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:03.964898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:03.968700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:03.968990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:03.969696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:03.970066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:03.978443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:04.003287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 842/1693] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:04.011730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:04.048861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 852/1694] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:04.055312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4537d80-9937-46e0-93a4-f51a9acd7692 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:04.064360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:04.069896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:04.082659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:04.083344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.083634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.083911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.084763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:04.086435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:04.115502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 843/1695] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:04.118735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:04.138364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9e35e-316b-4292-af05-b84363cc1788 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:04.162930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:04.165509 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.165836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.167274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] PATH_INFO: `/v3/projects/5c56388c09a24560962addd9ad5c888c/users/7a194e4be2614d8b9773f0a4b9f7d9ed/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.219124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c56388c09a24560962addd9ad5c888c, user_id=7a194e4be2614d8b9773f0a4b9f7d9ed, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:04.231039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45073ea-201c-4dc6-bf53-02773d3ccf8f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:04.267384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 844/1696] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:04 2022] PUT /identity/v3/projects/5c56388c09a24560962addd9ad5c888c/users/7a194e4be2614d8b9773f0a4b9f7d9ed/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:04.277695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c176-4c2c-4d73-89e4-d8b4111ffc0f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.278387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c176-4c2c-4d73-89e4-d8b4111ffc0f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.278822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c176-4c2c-4d73-89e4-d8b4111ffc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.335135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2dc7c176-4c2c-4d73-89e4-d8b4111ffc0f None None] MFA Rules not processed for user `7a194e4be2614d8b9773f0a4b9f7d9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:04.336829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2dc7c176-4c2c-4d73-89e4-d8b4111ffc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:04.373351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a6d31-ae56-4f23-b5af-1e48dd980b7d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:04.395419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:04.521562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 845/1697] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:04.524217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.524520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.524803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.546289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:04.555391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:04.557399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.557686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.558040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.584354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] MFA Rules not processed for user `1c618f9ab8bd47a3a9913078f8ae895e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:04.585714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:04.714052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:04.715899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90d079-0763-4d30-97e8-534db154540e service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:04.734836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 853/1698] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:04.736517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:04.738201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:04.738701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:04.814997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] MFA Rules not processed for user `785140aa617e46608e8b3cf89aac5a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:04.820314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:04.860405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 846/1699] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:04.876886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.058926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 854/1700] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:05.073796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.080418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:05.121670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d82c93-b1db-4ae6-8a0c-b7aafd22437f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.140315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42c781-8641-4e5b-ba68-a2395d2a1328 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.147754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.148487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.151213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.151395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.152065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.158723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.167940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.168891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.170685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.175293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.215941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.216345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 847/1701] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:04 2022] POST /identity/v3/projects => generated 409 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:05.217862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.218042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.250569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 855/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:05.385489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.408331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302bb87-0292-4a08-b41f-cce85dc1e028 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.421796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e436ea4-3e25-4039-9e06-bb7aec844b14 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.428271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.429042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.429397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.430054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.431249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.434168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.461687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 848/1703] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:05 2022] POST /identity/v3/users => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:05.474874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:05.480565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.493759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b073-7825-407b-9dca-ab8f5d21f37b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.503530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.505636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.506075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.507134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.509051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.513055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.520346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:05.551715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 849/1704] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:05.565998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.601794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8b79a-c4bc-428c-8b61-20578992a8ee admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.621952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.625016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.628129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.628577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] PATH_INFO: `/v3/projects/798a8b20c04d4d8daadefcfacdbadee5/users/cae08acdacd647839cad09041559a40c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.634637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.636209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.639548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.640037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.682593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=798a8b20c04d4d8daadefcfacdbadee5, user_id=cae08acdacd647839cad09041559a40c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.690097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.719882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 850/1705] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:05 2022] PUT /identity/v3/projects/798a8b20c04d4d8daadefcfacdbadee5/users/cae08acdacd647839cad09041559a40c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:05.725753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.822551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.824459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.839131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 856/1706] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:05.922318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cba1089-c63f-421d-afd5-88b825d8797d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:05.927506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:05.928560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:05.928990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:05.941599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:05.942541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:05.945343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b54b3-07d1-48f4-8a93-20cf14510163 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:05.968039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:05.978184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 851/1707] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:05.984915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfa62e-16ed-40cf-bf9f-90ee3f2c9332 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:06.000316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:06.002069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:06.003019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:06.003494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] PATH_INFO: `/v3/projects/798a8b20c04d4d8daadefcfacdbadee5/users/cae08acdacd647839cad09041559a40c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:06.015938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=798a8b20c04d4d8daadefcfacdbadee5, user_id=cae08acdacd647839cad09041559a40c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:06.066697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:06.120489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 857/1708] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:05 2022] PUT /identity/v3/projects/798a8b20c04d4d8daadefcfacdbadee5/users/cae08acdacd647839cad09041559a40c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:06.133603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:06.134715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:06.135143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:06.198762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] MFA Rules not processed for user `cae08acdacd647839cad09041559a40c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:06.201430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:06.480751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 852/1709] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:06 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:06.481578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:06.483513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:06.657908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05cc18-8c7c-45de-95eb-f7d7a75d4461 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:06.663254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:06.664168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:06.664609 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:06.665047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:06.677790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:06.679603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:06.701025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 858/1710] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:06 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:07.247272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:07.249370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:07.273189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692288d-bea5-46c3-a404-5723996f5180 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:07.282441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:07.291193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:07.291713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:07.292523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:07.461990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:07.463514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:07.499931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:07.507387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:07.513855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763030b3-236c-4367-bd15-8079e79e4d9d service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:07.558939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 853/1711] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:07.653832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:07.713690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76246e3a-f263-471d-8571-1d8bee035c27 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:07.731878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:07.736866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:07.737161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:07.741843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] PATH_INFO: `/v3/users/c3050a754d2e4ddc8d43aa397873e827` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:07.743668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3050a754d2e4ddc8d43aa397873e827)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:07.753917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:07.860405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3050a754d2e4ddc8d43aa397873e827'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:07.876674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3050a754d2e4ddc8d43aa397873e827'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:07.880325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3050a754d2e4ddc8d43aa397873e827'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:07.910238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3050a754d2e4ddc8d43aa397873e827'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:07.945116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3050a754d2e4ddc8d43aa397873e827'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:07.956327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 859/1712] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:07 2022] DELETE /identity/v3/users/c3050a754d2e4ddc8d43aa397873e827 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:07.960996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215607af-453f-44a0-8082-bf15ffb07d64 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:07.961294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215607af-453f-44a0-8082-bf15ffb07d64 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:07.961633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215607af-453f-44a0-8082-bf15ffb07d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:08.002464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-215607af-453f-44a0-8082-bf15ffb07d64 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:08.003771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-215607af-453f-44a0-8082-bf15ffb07d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:08.292788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 854/1713] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:08.433250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:08.434969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:08.609965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69540cf-d6c7-4968-a1cb-c03b8df69682 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:08.620664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:08.621402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:08.623332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:08.624209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:08.644940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:08.647203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c367c-9fe0-4a7b-82dd-1e95000d33ce service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:08.668427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 860/1714] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:09.258469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309776c-6406-4513-bb13-045714c57132 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:09.258951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309776c-6406-4513-bb13-045714c57132 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:09.260787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309776c-6406-4513-bb13-045714c57132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:09.293916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0309776c-6406-4513-bb13-045714c57132 None None] MFA Rules not processed for user `af0dee4e1dfb4f879dfe41622a9e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:09.300659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0309776c-6406-4513-bb13-045714c57132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:09.514062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1ccb1-69d7-4ee1-a2ad-1e0d5c0cf08a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:09.514566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1ccb1-69d7-4ee1-a2ad-1e0d5c0cf08a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:09.514665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1ccb1-69d7-4ee1-a2ad-1e0d5c0cf08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:09.523046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 855/1715] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:09.537401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cbd1ccb1-69d7-4ee1-a2ad-1e0d5c0cf08a None None] MFA Rules not processed for user `af0dee4e1dfb4f879dfe41622a9e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:09.539062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cbd1ccb1-69d7-4ee1-a2ad-1e0d5c0cf08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:09.552009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 861/1716] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:09.560166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309776c-6406-4513-bb13-045714c57132 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:09.562425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0309776c-6406-4513-bb13-045714c57132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:09.800289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309776c-6406-4513-bb13-045714c57132 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:09.817630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:09.818445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:09.818804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:09.819144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:09.845051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:09.847214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84073c-c10c-4052-92be-0d83c9c7a514 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:09.885395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 856/1717] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:10.455841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4030-32d0-49a5-8118-b9d4d2c78d75 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:10.459104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4030-32d0-49a5-8118-b9d4d2c78d75 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:10.459247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4030-32d0-49a5-8118-b9d4d2c78d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:10.530496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-df7b4030-32d0-49a5-8118-b9d4d2c78d75 None None] MFA Rules not processed for user `b974bab2e7934b57ab79128210c439a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:10.532672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df7b4030-32d0-49a5-8118-b9d4d2c78d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:10.640766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e34e-5758-426b-a503-daab077f0158 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:10.641606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e34e-5758-426b-a503-daab077f0158 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:10.642065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887e34e-5758-426b-a503-daab077f0158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:10.655956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:10.675837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7887e34e-5758-426b-a503-daab077f0158 None None] MFA Rules not processed for user `f66183ca9d884f43befaace02099c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:10.682001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7887e34e-5758-426b-a503-daab077f0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:10.684419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 862/1718] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:10.689871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:10.701457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:10.710157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:10.710736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:10.751206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] MFA Rules not processed for user `b974bab2e7934b57ab79128210c439a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:10.770034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:10.791383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 863/1719] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:10.802439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:10.804570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:10.835326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2585a-69c4-4c07-8aa5-2cc246671171 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:10.848375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:10.849885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:10.850185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:10.850466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:10.862529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 857/1720] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:10.869245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:10.873589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d1988-ad2f-4c46-9210-dbc60106b3dc service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:10.901276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887e34e-5758-426b-a503-daab077f0158 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:10.902746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7887e34e-5758-426b-a503-daab077f0158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:10.903414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 864/1721] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:10.950977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7887e34e-5758-426b-a503-daab077f0158 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:10.961239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:10.962437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:10.962728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:10.963012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] PATH_INFO: `/v3/projects/3b660d743d01487fa2e57f14b631de75` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:10.964350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b660d743d01487fa2e57f14b631de75)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:10.966528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:11.087665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6bc4a-e282-4d11-b45d-163e73be6a56 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.087892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6bc4a-e282-4d11-b45d-163e73be6a56 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.088220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c6bc4a-e282-4d11-b45d-163e73be6a56 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.089770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 865/1722] 10.222.0.23 () {54 vars in 912 bytes} [Mon Jun 20 07:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:11.095741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.096315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.096739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.113141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b660d743d01487fa2e57f14b631de75', 'request_id': 'req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.126632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b660d743d01487fa2e57f14b631de75', 'request_id': 'req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.141383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b660d743d01487fa2e57f14b631de75', 'request_id': 'req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.183364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] The token cache is being invalidate because project 3b660d743d01487fa2e57f14b631de75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:11.183669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.186752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.193911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 858/1723] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:10 2022] DELETE /identity/v3/projects/3b660d743d01487fa2e57f14b631de75 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:11.196299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:11.201544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:11.203493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:11.204915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:11.322988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b9b29-09e7-4714-9bb1-a3ce7efbc3fb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:11.332795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:11.333744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.334031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.334308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] PATH_INFO: `/v3/users/b9372cd583db4107903c4f95f713690b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.335664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9372cd583db4107903c4f95f713690b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:11.337808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:11.361903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 866/1724] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:07:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:11.376893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:11.378484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:11.390949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4054a4-b23f-4a34-bf9e-50d445901963 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:11.395197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:11.395872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.396191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.396497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.406225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:11.407835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530310dd-304a-402c-9deb-0e083b825f8f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:11.418200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 867/1725] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:11.420403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302324aa-c38f-4129-8a63-3ac28a5801e5 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.421376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302324aa-c38f-4129-8a63-3ac28a5801e5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.421687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302324aa-c38f-4129-8a63-3ac28a5801e5 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.423794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 868/1726] 10.222.0.23 () {54 vars in 912 bytes} [Mon Jun 20 07:07:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:11.430887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.431564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.431927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.445279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:11.459443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9372cd583db4107903c4f95f713690b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.466994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9372cd583db4107903c4f95f713690b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.470096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9372cd583db4107903c4f95f713690b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.484230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:11.485501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9372cd583db4107903c4f95f713690b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.487694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:11.495953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 869/1727] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:07:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:11.521684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03fa8b1c-6964-48ac-b7f5-b7dc3d0389a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9372cd583db4107903c4f95f713690b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:11.533043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:11.535767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:11.537939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 859/1728] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:11 2022] DELETE /identity/v3/users/b9372cd583db4107903c4f95f713690b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:11.555000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebee8e-146e-42b2-a32e-c352fbcf608f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:11.727951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:11.728662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:11.728952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:11.729683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:11.762591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:11.764136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:11.802353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 870/1729] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:12.489608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af790d9-07e5-45ba-b53c-fa407b239767 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:12.490180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af790d9-07e5-45ba-b53c-fa407b239767 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:12.490180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af790d9-07e5-45ba-b53c-fa407b239767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:12.532751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8af790d9-07e5-45ba-b53c-fa407b239767 None None] MFA Rules not processed for user `785140aa617e46608e8b3cf89aac5a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:12.536669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8af790d9-07e5-45ba-b53c-fa407b239767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:12.559216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:12.561964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:12.688937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 860/1730] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:12.706815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2831-c92b-4413-93f7-c6d82f33f6d2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:12.706815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2831-c92b-4413-93f7-c6d82f33f6d2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:12.710018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e2831-c92b-4413-93f7-c6d82f33f6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:12.721643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:12.731584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a792f-485d-4679-bb67-31cd238350a2 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:12.741245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:12.742475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:12.742917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:12.743274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] PATH_INFO: `/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:12.745120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0a6dcaad3a419bb863fb263d7f8292)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:12.752003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:12.778065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-018e2831-c92b-4413-93f7-c6d82f33f6d2 None None] MFA Rules not processed for user `1c618f9ab8bd47a3a9913078f8ae895e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:12.779817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-018e2831-c92b-4413-93f7-c6d82f33f6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:12.847026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0a6dcaad3a419bb863fb263d7f8292', 'request_id': 'req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:12.880319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0a6dcaad3a419bb863fb263d7f8292', 'request_id': 'req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:12.892152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0a6dcaad3a419bb863fb263d7f8292', 'request_id': 'req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:12.908296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 861/1731] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:12.911109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308881-4256-462c-8baf-819f43e705fb None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:12.911782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308881-4256-462c-8baf-819f43e705fb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:12.912531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d308881-4256-462c-8baf-819f43e705fb None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:12.914693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 862/1732] 10.222.0.23 () {54 vars in 960 bytes} [Mon Jun 20 07:07:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:12.917052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] The token cache is being invalidate because project 3a0a6dcaad3a419bb863fb263d7f8292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:12.917421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:12.918297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3ac960e5-fe9a-47aa-9df4-8d0f22e0b7cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:12.924870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308881-4256-462c-8baf-819f43e705fb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:12.927306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1d308881-4256-462c-8baf-819f43e705fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:12.934628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 871/1733] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:12 2022] DELETE /identity/v3/projects/3a0a6dcaad3a419bb863fb263d7f8292 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:12.936497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2dac-54f5-45fd-960e-f448fdc1bba7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:12.936935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2dac-54f5-45fd-960e-f448fdc1bba7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:12.937224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2dac-54f5-45fd-960e-f448fdc1bba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:12.995300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8d9e2dac-54f5-45fd-960e-f448fdc1bba7 None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:12.996575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8d9e2dac-54f5-45fd-960e-f448fdc1bba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.071718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d308881-4256-462c-8baf-819f43e705fb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.082916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:13.085695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.086107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.086687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.090071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.112316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 872/1734] 10.222.0.23 () {58 vars in 1057 bytes} [Mon Jun 20 07:07:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:13.113968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.114468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.117071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.129786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:13.132848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:13.135423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f4d59-400f-47db-a005-97fb66e9008b service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:13.157926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:13.161336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.173052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 863/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:13.176343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92c585-74dd-47a0-aa3b-fb758fb1f7ec None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.177159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92c585-74dd-47a0-aa3b-fb758fb1f7ec None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.179476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92c585-74dd-47a0-aa3b-fb758fb1f7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.277803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3b92c585-74dd-47a0-aa3b-fb758fb1f7ec None None] MFA Rules not processed for user `af0dee4e1dfb4f879dfe41622a9e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:13.281261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3b92c585-74dd-47a0-aa3b-fb758fb1f7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.469833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 873/1736] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:13.471349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:13.490074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 864/1737] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:13.492364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.492822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.493257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.517349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d7087-15a8-491f-b61f-e613ccc8e149 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.536536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:13.538045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.538045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.538045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.544317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:13.548682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.562125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:13.569121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:13.579434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 865/1738] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:13.591136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:13.596295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.647061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 874/1739] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:13.649926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:13.657043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb8ef9-2eb4-4891-875b-baf67cfeee1f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.671461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:13.672389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.672679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.673421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] PATH_INFO: `/v3/users/6cf35ec82546401f9bb339741ea6adee` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.686720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cf35ec82546401f9bb339741ea6adee)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:13.688709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:13.705783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c26e1-92a7-4aed-9c39-c2c92bd32096 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.725828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:13.727166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.727450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.727731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.732976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:13.733290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:13.734923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:13.746310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:13.796097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 875/1740] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:13.798407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:13.814407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea300c88-6d65-49a9-acfb-246be311e1d5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.822926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:13.824194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.824777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.825266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.826047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cf35ec82546401f9bb339741ea6adee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:13.827673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:13.831012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:13.832638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cf35ec82546401f9bb339741ea6adee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:13.835346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cf35ec82546401f9bb339741ea6adee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:13.854092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cf35ec82546401f9bb339741ea6adee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:13.876732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 876/1741] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:13.878772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:13.881173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:13.882205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-212b513d-68e3-458f-b1cc-dbf11b85203a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cf35ec82546401f9bb339741ea6adee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:13.913089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 866/1742] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:13 2022] DELETE /identity/v3/users/6cf35ec82546401f9bb339741ea6adee => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:13.926581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:13.926854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:13.927150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:13.971215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca2b4-3b62-439b-ab24-f04f9ca50318 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:13.979697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:13.983646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:14.194911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:14.198714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 867/1743] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:14.201079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.201512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.201827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:14.202521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:14.219587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:14.231060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:14.234426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:14.275235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:14.306557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 877/1744] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:13 2022] POST /identity/v3/users => generated 292 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:14.311226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:14.348434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9c8a8-4bec-459a-ac75-0a74cbf275f1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:14.394674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:14.408559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.416520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.420420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:14.431575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:14.442291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:14.486079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bee4ef-784b-48e0-a81e-d715ef8bf477 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:14.502593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:14.508494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.509038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.510746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:14.534935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 878/1745] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:14.535842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:14.543528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:14.550179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:14.592312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279d350-e439-4346-b545-1f5250224970 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:14.593263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 868/1746] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:14.624304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:14.626224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.627367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.629672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:14.630633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:14.633706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ecfbe-3e62-4518-83bb-485f70e46071 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:14.689924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 879/1747] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:14.695316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:14.723350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed4af5-b693-4ccd-bdc7-7a73468b690a service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:14.748980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:14.756317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.756918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.758361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] PATH_INFO: `/v3/projects/efbe8fb31f604395be4968ed1f489f9f/users/05a235c320cc41099829e8f58b03b642/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:14.846809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efbe8fb31f604395be4968ed1f489f9f, user_id=05a235c320cc41099829e8f58b03b642, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:14.853797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:14.907904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 869/1748] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:14 2022] PUT /identity/v3/projects/efbe8fb31f604395be4968ed1f489f9f/users/05a235c320cc41099829e8f58b03b642/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:14.917923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:14.918891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:14.919183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:15.000771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] MFA Rules not processed for user `05a235c320cc41099829e8f58b03b642`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:15.005208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:15.130091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:15.240437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 880/1749] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:15.241245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:15.506044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:15.508312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:15.538081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33c48-2adb-4453-b070-e1c14df26354 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:15.554945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ed16f-68f3-4f9b-9ae5-0383fc3f8940 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:15.561322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:15.562491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:15.562966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:15.566902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] PATH_INFO: `/v3/projects/af55321b0ab5484399028d8cf04ae1bf` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:15.569345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=af55321b0ab5484399028d8cf04ae1bf)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:15.572210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:15.597857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:15.607870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:15.608314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:15.608721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:15.612690 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:15.619872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:15.736945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af55321b0ab5484399028d8cf04ae1bf', 'request_id': 'req-e48e1533-72d7-4706-9c54-301b0ca9bbbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:15.753774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af55321b0ab5484399028d8cf04ae1bf', 'request_id': 'req-e48e1533-72d7-4706-9c54-301b0ca9bbbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:15.766940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 870/1750] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:15 2022] POST /identity/v3/projects => generated 385 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:15.770543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:15.776212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af55321b0ab5484399028d8cf04ae1bf', 'request_id': 'req-e48e1533-72d7-4706-9c54-301b0ca9bbbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:15.824694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] The token cache is being invalidate because project af55321b0ab5484399028d8cf04ae1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:15.825004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:15.834732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e48e1533-72d7-4706-9c54-301b0ca9bbbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:15.845656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 881/1751] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:15 2022] DELETE /identity/v3/projects/af55321b0ab5484399028d8cf04ae1bf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:15.852914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21508ac-210b-451e-927e-c08b17c4017e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:15.853575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21508ac-210b-451e-927e-c08b17c4017e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:15.854143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21508ac-210b-451e-927e-c08b17c4017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:15.917417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e21508ac-210b-451e-927e-c08b17c4017e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:15.920157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e21508ac-210b-451e-927e-c08b17c4017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:16.007318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59524-21a1-40c9-ad2c-a8dc4fb401cd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.025018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.026187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.026930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.027471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.029594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.035566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.088475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:16.106687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 882/1752] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:16.111100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 871/1753] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:15 2022] POST /identity/v3/users => generated 299 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:16.114132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.114699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.115115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.117220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.121399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:16.154329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:16.156828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:16.160212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf184d3-e922-4de0-8956-e69cb884a37d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.177158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 883/1754] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:16.180412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.181065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.181819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.182367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.182769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.183786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.186165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.205581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 872/1755] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:16.206673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.207435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d16c7-f3a3-4b05-b75f-4fbbda11b173 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.218468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b60e583-f816-4580-b1ce-d48b911f29b0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.223022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.223749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.224513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.224847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.225156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.225464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] PATH_INFO: `/v3/projects/e4bf96048b4c40118a630beb7c56dd73/users/f3461710482c482a81c57e280ebed680/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.225867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.226327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.229450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:16.230126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.234795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.251445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4bf96048b4c40118a630beb7c56dd73, user_id=f3461710482c482a81c57e280ebed680, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.256078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.258440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 884/1756] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:16.262208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.306348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 873/1757] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:16 2022] PUT /identity/v3/projects/e4bf96048b4c40118a630beb7c56dd73/users/f3461710482c482a81c57e280ebed680/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:16.312538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.329053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5071f52-066c-43c0-b4a4-99fc93434e2f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.505547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:16.576285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.576527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.579424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.579917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.580839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.581622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d94a59-b4c3-41aa-bba1-ff6f8fbb3642 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.585333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.618162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.620595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.620889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.621181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.621944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.625843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.665708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 874/1758] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:16.668045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.674309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 885/1759] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:16 2022] POST /identity/v3/projects => generated 355 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:16.678442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:16.703512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c8ee-3c89-4d5a-8603-86b477ff83a8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.913993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.916393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.916523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.916785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] PATH_INFO: `/v3/projects/e4bf96048b4c40118a630beb7c56dd73/users/f3461710482c482a81c57e280ebed680/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.923886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4bf96048b4c40118a630beb7c56dd73, user_id=f3461710482c482a81c57e280ebed680, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.927008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3067ae9-fb49-4c82-a73f-76ca01f2c9db admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:16.931657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c196a232-3314-4df9-ab0a-76178bea22b9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.949818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:16.951061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.954061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 875/1760] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:16 2022] PUT /identity/v3/projects/e4bf96048b4c40118a630beb7c56dd73/users/f3461710482c482a81c57e280ebed680/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:16.955756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.956056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:16.956845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:16.958831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:16.975598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:16.981272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:16.981589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:17.028089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 886/1761] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:16 2022] POST /identity/v3/users => generated 278 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:17.033278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:17.061876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c920206-76ec-4b09-b221-64a428848471 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:17.075436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] MFA Rules not processed for user `f3461710482c482a81c57e280ebed680`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:17.080925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:17.286019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:17.356471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:17.366839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:17.367199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:17.367484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:17.368306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:17.369965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:17.371223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 876/1762] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:17.389987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:17.392677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:17.416204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 887/1763] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:17.422680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:17.465598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7593-3f4d-46a8-8df7-505c9928899f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:17.525100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:17.525802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:17.526303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:17.526407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:17.527106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:17.542983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:17.624513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 888/1764] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:17.625529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:17.640650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:17.704577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4beb4-9673-4675-98f5-34e1009359e8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:17.732463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:17.733375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:17.733663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:17.733944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] PATH_INFO: `/v3/users/7a194e4be2614d8b9773f0a4b9f7d9ed` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:17.734855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c3458-a231-4e64-8657-13556ae9a1f3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:17.753037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a194e4be2614d8b9773f0a4b9f7d9ed)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:17.755579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:17.755730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:17.756008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:17.756327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:17.762883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:17.769580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:17.948547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:17.995844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a194e4be2614d8b9773f0a4b9f7d9ed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:18.038459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a194e4be2614d8b9773f0a4b9f7d9ed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:18.040316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:18.043827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a194e4be2614d8b9773f0a4b9f7d9ed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:18.048889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:18.086703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a194e4be2614d8b9773f0a4b9f7d9ed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:18.104169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 877/1765] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:18.111617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:18.162714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79f378-ddf9-430e-86d2-d200c0c09c23 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:18.194085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:18.196964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4a695bdd-9b17-4006-b088-0c6c14fe109c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a194e4be2614d8b9773f0a4b9f7d9ed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:18.199802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:18.205589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:18.208863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] PATH_INFO: `/v3/projects/2288c8a2ee544658b0cf357a68c581ea/users/0be762ddf0cb4d788ed56a5c649477bf/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:18.244558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 889/1766] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:17 2022] DELETE /identity/v3/users/7a194e4be2614d8b9773f0a4b9f7d9ed => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:18.251354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cc03-c07b-4bf2-ae27-8889e5c539be None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:18.251646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cc03-c07b-4bf2-ae27-8889e5c539be None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:18.251926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81cc03-c07b-4bf2-ae27-8889e5c539be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:18.320678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2288c8a2ee544658b0cf357a68c581ea, user_id=0be762ddf0cb4d788ed56a5c649477bf, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:18.324746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bf81cc03-c07b-4bf2-ae27-8889e5c539be None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:18.326824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bf81cc03-c07b-4bf2-ae27-8889e5c539be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:18.335623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc327ae1-a866-4882-866b-345adab09e59 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:18.409201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 878/1767] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:18 2022] PUT /identity/v3/projects/2288c8a2ee544658b0cf357a68c581ea/users/0be762ddf0cb4d788ed56a5c649477bf/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:18.427352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:18.431093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:18.434980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:18.540031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:18.551097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:18.822544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 890/1768] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 574 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:18.824480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:18.827871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:18.830816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:18.900145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 879/1769] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:18.912866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:18.916944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:18.959274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] MFA Rules not processed for user `0be762ddf0cb4d788ed56a5c649477bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:18.976424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:19.051465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:19.197861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842a718-5dfa-4a3a-b322-ab02ae9e4856 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.219645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.220725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.221236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.221750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.399436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 891/1770] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 576 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:19.400191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:19.402578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.404703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.414496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:19.418869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 880/1771] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:19.419972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:19.421430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:19.441186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176120d3-5f77-4b46-8462-903a1924cb05 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.448066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.449084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.449723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.450280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.451782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.455399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.508693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 881/1772] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:19 2022] POST /identity/v3/projects => generated 357 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:19.510809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:19.515298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:19.577675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967ef7e-78ed-4701-ba16-5c803f53aced None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.725657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d348fe-bf84-4778-ba95-18f1294e8f43 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.738746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.740876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.741908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.742345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.745439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.745671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.746500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.747068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.748075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.749826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.807941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 882/1773] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:19 2022] POST /identity/v3/users => generated 285 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:19.809143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:19.833219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47380ae-fa07-4249-825c-8b9c1604cb1c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.835323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.836293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.847413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.848022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.848520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.848858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.849744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.852290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.875206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 883/1774] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:19.880054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:19.887980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 892/1775] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:07:19.901615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232876a0-5e6f-4eef-a6c6-87813dcfc8c7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:19.920842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:19.921803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:19.922084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:19.922363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] PATH_INFO: `/v3/projects/2df213a09db341648bf078bcfffc639a/users/5242d9f4d4694b1ca9c5fed76b473106/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:19.963197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df213a09db341648bf078bcfffc639a, user_id=5242d9f4d4694b1ca9c5fed76b473106, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:19.965688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:19.990666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:20.002794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 884/1776] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:19 2022] PUT /identity/v3/projects/2df213a09db341648bf078bcfffc639a/users/5242d9f4d4694b1ca9c5fed76b473106/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:20.005051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:20.023735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a8b39f-0876-479d-ab47-50c1fa950874 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:20.233171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:20.240380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:20.240974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:20.242957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:20.244257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:20.248906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d045be-4d82-4562-956a-38b6276c1789 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:20.292824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 893/1777] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:20.293375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:20.315062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8810e-7f58-439b-a7c3-466fa9426630 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:20.332915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:20.336807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:20.337138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:20.337450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] PATH_INFO: `/v3/projects/2df213a09db341648bf078bcfffc639a/users/5242d9f4d4694b1ca9c5fed76b473106/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:20.348045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df213a09db341648bf078bcfffc639a, user_id=5242d9f4d4694b1ca9c5fed76b473106, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:20.362061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:20.414366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 885/1778] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:20 2022] PUT /identity/v3/projects/2df213a09db341648bf078bcfffc639a/users/5242d9f4d4694b1ca9c5fed76b473106/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:20.427653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:20.428102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:20.428483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:20.489135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] MFA Rules not processed for user `5242d9f4d4694b1ca9c5fed76b473106`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:20.490824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:20.530261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:20.844598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 894/1779] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:20 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:20.857491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:20.859902 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:21.005658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b40379-d189-4ffe-851d-a14ef67dc435 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:21.017001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:21.018182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:21.018621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:21.019088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:21.064290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:21.065880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:21.105816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 886/1780] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:20 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:21.320807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:21.332434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:21.333733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb3198-6351-434c-9d0a-8a3f748cbe48 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:21.334419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:21.348481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:21.349099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:21.349390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:21.349673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:21.472703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da050bb2-b896-44c7-8e77-5b47f4ac3dc6 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:21.502840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:21.503933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:21.504418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:21.504854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] PATH_INFO: `/v3/projects/5c56388c09a24560962addd9ad5c888c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:21.506946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c56388c09a24560962addd9ad5c888c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:21.513608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:21.564989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:21.573005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:21.635060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 895/1781] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:21 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:21.662223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c56388c09a24560962addd9ad5c888c', 'request_id': 'req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:21.690912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c56388c09a24560962addd9ad5c888c', 'request_id': 'req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:21.725432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c56388c09a24560962addd9ad5c888c', 'request_id': 'req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:21.753103 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] The token cache is being invalidate because project 5c56388c09a24560962addd9ad5c888c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:21.754099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:21.756316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dcf31bbf-952e-4acd-9ed4-f14ed230aae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:21.758322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 887/1782] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:21 2022] DELETE /identity/v3/projects/5c56388c09a24560962addd9ad5c888c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:21.762398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:21.791872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1e3da-2c00-4457-a0f6-01b039f94f68 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:22.065012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:22.065919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:22.066233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:22.066546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] PATH_INFO: `/v3/users/cae08acdacd647839cad09041559a40c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:22.076550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cae08acdacd647839cad09041559a40c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:22.078602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:22.276716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cae08acdacd647839cad09041559a40c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:22.284268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:22.322288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cae08acdacd647839cad09041559a40c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:22.323087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cae08acdacd647839cad09041559a40c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:22.335738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cae08acdacd647839cad09041559a40c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:22.364293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cae08acdacd647839cad09041559a40c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:22.376187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 896/1783] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:21 2022] DELETE /identity/v3/users/cae08acdacd647839cad09041559a40c => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:24.899595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:24.900182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:24.900324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:25.014544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:25.018951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:25.061206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:25.237470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 888/1784] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Jun 20 07:07:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:25.357494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:25.360723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:25.437421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:25.439319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:25.480093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8d5a4-1d1c-4364-827f-82241d29d583 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:25.499151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:25.501045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:25.504066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:25.506725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:25.546520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:25.550276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:25.588210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 889/1785] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:25.589127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:25.610867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1a73c-7e12-44ff-9a17-49b2dc9fc48c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:25.625178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:25.626326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:25.631022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:25.631457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:25.644823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:25.646448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:25.649715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a8346-2e7f-4b2c-aec6-13447a02e7d9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:25.676854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:25.677532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:25.677879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:25.682253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:25.683597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 890/1786] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:25.698768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:25.762602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:25.791922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a01f9-4d26-44f3-bafb-41300a07a87c service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:25.799091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:25.800865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:25.810425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:25.811450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:25.811617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:25.811949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:25.830736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 897/1787] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:25.850187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:25.850533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbca9b5-c2e3-41d2-a4c0-20d8bd1a9416 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:25.880296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 891/1788] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:27.865101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:27.867917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:28.045312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cd845-d2d8-4afc-8bf4-88c0eb4ff7c0 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:28.059783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:28.061036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:28.061491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:28.061876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] PATH_INFO: `/v3/projects/798a8b20c04d4d8daadefcfacdbadee5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:28.063367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798a8b20c04d4d8daadefcfacdbadee5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:28.067459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:28.119748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:28.195078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798a8b20c04d4d8daadefcfacdbadee5', 'request_id': 'req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:28.288145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798a8b20c04d4d8daadefcfacdbadee5', 'request_id': 'req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:28.315167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798a8b20c04d4d8daadefcfacdbadee5', 'request_id': 'req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:28.359038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] The token cache is being invalidate because project 798a8b20c04d4d8daadefcfacdbadee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:28.362568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:28.362568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3f1f463c-3b5a-400d-a45a-e1b4966a2aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:28.372025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:28.389172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:28.389172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:28.391920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 898/1789] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:27 2022] DELETE /identity/v3/projects/798a8b20c04d4d8daadefcfacdbadee5 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:28.434457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:28.438291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:28.669046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 892/1790] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:28.678195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:28.678806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:28.679193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:28.739906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:28.742438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:28.769324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 899/1791] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:28.769735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:28.787829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef1279-ffcd-4f38-861d-452e5ef95504 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:28.795562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:28.797281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:28.797988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:28.798265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:28.801695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:28.802023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:28.806195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:28.840367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 893/1792] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:28.842447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:28.869302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81a9e5-f19d-4917-a0c6-f38597b000a0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:28.890394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:28.892788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:28.893340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:28.893837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:28.896044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:28.901393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:28.964634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 900/1793] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:28 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:28.969387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:28.984894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bd23-63ac-4212-b9fd-d64fe4bf6e64 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:29.237302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:29.238044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:29.238332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:29.238616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:29.239342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:29.241345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:29.322225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 894/1794] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:28 2022] POST /identity/v3/users => generated 290 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:29.326305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:29.349853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4255e27-c5ca-4995-99a9-d2dc001b1eb3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:29.375330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:29.376045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:29.376357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:29.376636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:29.377378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:29.384736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:29.422862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:29.430700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:29.434789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 901/1795] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:29.443388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3fd2f-6e07-4c48-8163-04e80091d6fc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:29.476046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:29.477075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:29.477378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:29.477662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:29.478408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:29.480047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a5134-d760-49a5-b40e-12d3cbe6ceac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:29.510711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 895/1796] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:29.515700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:29.552936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a003d8-4de7-4413-b8d7-d85724307bf9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:29.572065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:29.592421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:29.593272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:29.593805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] PATH_INFO: `/v3/projects/2e559d0cdd1d4fa0afefe9f8f17488b0/users/46463be2f87b454fb13d50805bb7ffbc/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:29.673127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e559d0cdd1d4fa0afefe9f8f17488b0, user_id=46463be2f87b454fb13d50805bb7ffbc, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:29.675821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:29.742465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 902/1797] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:29 2022] PUT /identity/v3/projects/2e559d0cdd1d4fa0afefe9f8f17488b0/users/46463be2f87b454fb13d50805bb7ffbc/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:29.746110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:29.746400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:29.748009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:29.836431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] MFA Rules not processed for user `46463be2f87b454fb13d50805bb7ffbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:29.838135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:30.041244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 896/1798] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:30.046510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:30.255953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5f996-5629-4551-a17d-e75e1423e506 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:30.275764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:30.276484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:30.276822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:30.277111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:30.279014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:30.283844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:30.357779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 903/1799] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:30 2022] POST /identity/v3/projects => generated 381 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:30.359049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:30.374896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9257d4c1-8b4c-466f-9e9f-6ab700e4a0fe None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:30.575012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:30.576232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:30.576391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:30.576765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:30.577557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:30.583055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:30.627049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 897/1800] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:30 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:30.628250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:30.646267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c22415-d7d3-447f-99ea-f6f05a11577a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:30.675108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:30.675820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:30.676148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:30.680581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:30.681379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:30.683034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:30.793035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 904/1801] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:30.797464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:30.814196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0edfa8-f141-4fa5-8918-b11ffc417502 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:30.839716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:30.840893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:30.841184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:30.841466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] PATH_INFO: `/v3/projects/82e6c9d4aab247689623094381fee3e3/users/896fa08c0f584639980f0d2651050c1d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:30.893837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e6c9d4aab247689623094381fee3e3, user_id=896fa08c0f584639980f0d2651050c1d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:30.897166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:30.972195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 898/1802] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:30 2022] PUT /identity/v3/projects/82e6c9d4aab247689623094381fee3e3/users/896fa08c0f584639980f0d2651050c1d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:30.972448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:30.990769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5229188-ea9d-429c-b7c8-7a4c3b0324b7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:31.083350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:31.314473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:31.315357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:31.315894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:31.316221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:31.316997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:31.319011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21970d-39ab-4ddf-ab37-2b8d627e1b2b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:31.351510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 905/1803] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:31.352863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:31.374031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf5f96-0dbd-4447-a0e7-3adb55a97e05 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:31.386550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25576-ee68-43a5-ad51-d902aff7019c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:31.386845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25576-ee68-43a5-ad51-d902aff7019c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:31.387125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd25576-ee68-43a5-ad51-d902aff7019c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:31.388866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 906/1804] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:31.401302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:31.402037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:31.402412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:31.402699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] PATH_INFO: `/v3/projects/82e6c9d4aab247689623094381fee3e3/users/896fa08c0f584639980f0d2651050c1d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:31.408361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e6c9d4aab247689623094381fee3e3, user_id=896fa08c0f584639980f0d2651050c1d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:31.414149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:31.481100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 899/1805] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:31 2022] PUT /identity/v3/projects/82e6c9d4aab247689623094381fee3e3/users/896fa08c0f584639980f0d2651050c1d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:31.492775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f4ea-fe31-41b1-b10a-402a64de4f29 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:31.493280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f4ea-fe31-41b1-b10a-402a64de4f29 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:31.493574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908f4ea-fe31-41b1-b10a-402a64de4f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:31.596045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9908f4ea-fe31-41b1-b10a-402a64de4f29 None None] MFA Rules not processed for user `896fa08c0f584639980f0d2651050c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:31.597861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9908f4ea-fe31-41b1-b10a-402a64de4f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:31.850089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 907/1806] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:31.869257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:31.871079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:31.975743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:32.027722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8affd-72b9-438d-a19f-c1cca00062dc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:32.034141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:32.035056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:32.035504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:32.036189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:32.059153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f68ee-71fe-44d8-a427-e7597bb55063 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:32.059454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f68ee-71fe-44d8-a427-e7597bb55063 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:32.059828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f68ee-71fe-44d8-a427-e7597bb55063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:32.060374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:32.062821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:32.087371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 900/1807] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:32.096054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-063f68ee-71fe-44d8-a427-e7597bb55063 None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:32.098242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-063f68ee-71fe-44d8-a427-e7597bb55063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:32.230422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 908/1808] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:07:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:32.265806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:32.268439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:32.469578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070e44d-8d54-4dd8-b8bf-ce6aba647907 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:32.482526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:32.484944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:32.488427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:32.488988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] PATH_INFO: `/v3/users/05a235c320cc41099829e8f58b03b642` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:32.494240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a235c320cc41099829e8f58b03b642)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:32.502195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:32.672638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a235c320cc41099829e8f58b03b642'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:32.676779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a235c320cc41099829e8f58b03b642'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:32.677838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a235c320cc41099829e8f58b03b642'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:32.697149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a235c320cc41099829e8f58b03b642'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:32.727180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a235c320cc41099829e8f58b03b642'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:32.767630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 901/1809] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:32 2022] DELETE /identity/v3/users/05a235c320cc41099829e8f58b03b642 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:32.771709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:32.772211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:32.772720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:32.803037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:32.804677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:32.850010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:32.961978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 909/1810] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:33.062907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:33.065499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:33.173382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:33.177890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:33.191583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd7074-4b35-4e7c-a797-4b1526f2ffb2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:33.198865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:33.205065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:33.206124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:33.206124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] PATH_INFO: `/v3/users/46463be2f87b454fb13d50805bb7ffbc` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:33.207439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46463be2f87b454fb13d50805bb7ffbc)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:33.210861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:33.287261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4101f8-1d66-491f-a6d4-7d1df7699202 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:33.303014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46463be2f87b454fb13d50805bb7ffbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:33.306292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:33.307313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:33.307796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:33.308455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:33.311995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46463be2f87b454fb13d50805bb7ffbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:33.313285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46463be2f87b454fb13d50805bb7ffbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:33.349615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:33.353898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa227f8-8de7-4604-8ed7-e5167c51bcee service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:33.363779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46463be2f87b454fb13d50805bb7ffbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:33.374378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:33.417245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 902/1811] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:33.421362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46463be2f87b454fb13d50805bb7ffbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:33.452499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:33.453005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:33.453364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:33.454906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 910/1812] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:33 2022] DELETE /identity/v3/users/46463be2f87b454fb13d50805bb7ffbc => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:33.520589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:33.530955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:33.531352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:33.532677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:33.843241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 903/1813] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:33.908014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dd41e-99f6-43cc-8da7-f3236ab7cb8e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:33.918488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:33.919238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:33.919524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:33.919811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:33.927298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:33.954500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:33.957913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6b053-eaec-46aa-9b83-d1301da130d0 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:33.977752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:33.979439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:34.021736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 911/1814] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:34.154454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353bd5c-8c06-408d-8101-83830cf5dda6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:34.164784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:34.173102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:34.173102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:34.173102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:34.193817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:34.196918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710bff3-dd91-4382-9ef4-dde900b553f7 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:34.224011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 904/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:34.671778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b9f9-a62e-4cb2-8339-4a1e5afb2b95 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:34.672890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b9f9-a62e-4cb2-8339-4a1e5afb2b95 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:34.673333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748b9f9-a62e-4cb2-8339-4a1e5afb2b95 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:34.675256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 912/1816] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:07:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:34.829669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:34.829970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:34.830480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:34.891747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:34.893288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:34.932534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 905/1817] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:07:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:35.450733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748b9f9-a62e-4cb2-8339-4a1e5afb2b95 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:35.476942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748b9f9-a62e-4cb2-8339-4a1e5afb2b95 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:35.491450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:35.492273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:35.492687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:35.493074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:35.506831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:35.509010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:35.547158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 913/1818] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:35.615841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:35.618410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:35.629077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:35.629077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:35.653456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78f36f-efb4-4fd9-b5c8-b0139994741f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:35.657987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09103117-ec53-49c4-a321-a7cc2cf8404c service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:35.673331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:35.674212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:35.674493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:35.674767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] PATH_INFO: `/v3/projects/efbe8fb31f604395be4968ed1f489f9f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:35.675985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efbe8fb31f604395be4968ed1f489f9f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:35.677941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:35.681336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:35.683008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:35.685320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:35.685617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] PATH_INFO: `/v3/projects/2e559d0cdd1d4fa0afefe9f8f17488b0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:35.686962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e559d0cdd1d4fa0afefe9f8f17488b0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:35.688978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:35.723578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:35.786582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e559d0cdd1d4fa0afefe9f8f17488b0', 'request_id': 'req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.805993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e559d0cdd1d4fa0afefe9f8f17488b0', 'request_id': 'req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.821769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e559d0cdd1d4fa0afefe9f8f17488b0', 'request_id': 'req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.842550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] The token cache is being invalidate because project 2e559d0cdd1d4fa0afefe9f8f17488b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:35.842849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.844098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.852073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 914/1819] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:35 2022] DELETE /identity/v3/projects/2e559d0cdd1d4fa0afefe9f8f17488b0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:35.853178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:35.857927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:35.940673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efbe8fb31f604395be4968ed1f489f9f', 'request_id': 'req-67b8d725-c37e-4de0-b293-d44c60456134'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.948570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efbe8fb31f604395be4968ed1f489f9f', 'request_id': 'req-67b8d725-c37e-4de0-b293-d44c60456134'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.958560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efbe8fb31f604395be4968ed1f489f9f', 'request_id': 'req-67b8d725-c37e-4de0-b293-d44c60456134'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.975200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] The token cache is being invalidate because project efbe8fb31f604395be4968ed1f489f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:35.975495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.976401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:35.977179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47d349-fcca-4fb9-a25a-6f315bb46ebb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:35.978536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 906/1820] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:35 2022] DELETE /identity/v3/projects/efbe8fb31f604395be4968ed1f489f9f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:35.984941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:35.987055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:36.046915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:36.143599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8d725-c37e-4de0-b293-d44c60456134 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:36.159112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:36.160696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:36.161167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:36.161637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] PATH_INFO: `/v3/users/f3461710482c482a81c57e280ebed680` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:36.163381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3461710482c482a81c57e280ebed680)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:36.165966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:36.216771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:36.217843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:36.218341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:36.218770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] PATH_INFO: `/v3/users/896fa08c0f584639980f0d2651050c1d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:36.221428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=896fa08c0f584639980f0d2651050c1d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:36.224021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:36.253258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3461710482c482a81c57e280ebed680'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.254309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3461710482c482a81c57e280ebed680'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.256006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3461710482c482a81c57e280ebed680'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.271633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3461710482c482a81c57e280ebed680'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.295308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3461710482c482a81c57e280ebed680'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.323446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 907/1821] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:35 2022] DELETE /identity/v3/users/f3461710482c482a81c57e280ebed680 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:36.360451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896fa08c0f584639980f0d2651050c1d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.362198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:36.372585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896fa08c0f584639980f0d2651050c1d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.374112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896fa08c0f584639980f0d2651050c1d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.392062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896fa08c0f584639980f0d2651050c1d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.411594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896fa08c0f584639980f0d2651050c1d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:36.421596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 915/1822] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:35 2022] DELETE /identity/v3/users/896fa08c0f584639980f0d2651050c1d => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:37.285383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.286896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.404775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb01997-c4f0-4306-9842-0b5480bfb01e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.421007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.422509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.465402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:37.491635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.493214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.493522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.494265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] PATH_INFO: `/v3/projects/e4bf96048b4c40118a630beb7c56dd73` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.498670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4bf96048b4c40118a630beb7c56dd73)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.499227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.534249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f9911c-3ffd-4c9e-86bb-b70e93fc3034 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.541856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.543010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.544658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.545468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] PATH_INFO: `/v3/projects/82e6c9d4aab247689623094381fee3e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.547087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e6c9d4aab247689623094381fee3e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.549044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.593241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4bf96048b4c40118a630beb7c56dd73', 'request_id': 'req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.601361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4bf96048b4c40118a630beb7c56dd73', 'request_id': 'req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.609158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4bf96048b4c40118a630beb7c56dd73', 'request_id': 'req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.624435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e6c9d4aab247689623094381fee3e3', 'request_id': 'req-6cca7a78-7dde-462c-a6ff-a8aabf9247da'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.625302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] The token cache is being invalidate because project e4bf96048b4c40118a630beb7c56dd73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:37.625772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.626498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4a681ed5-97af-4166-91b6-dfbc6b5db9b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.628021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 908/1823] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:37 2022] DELETE /identity/v3/projects/e4bf96048b4c40118a630beb7c56dd73 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:37.634162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e6c9d4aab247689623094381fee3e3', 'request_id': 'req-6cca7a78-7dde-462c-a6ff-a8aabf9247da'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.634673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6d4b-62cf-42dd-91b5-0ad1cfd3e818 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.634973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6d4b-62cf-42dd-91b5-0ad1cfd3e818 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.635258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d6d4b-62cf-42dd-91b5-0ad1cfd3e818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.643252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e6c9d4aab247689623094381fee3e3', 'request_id': 'req-6cca7a78-7dde-462c-a6ff-a8aabf9247da'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.656362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-df0d6d4b-62cf-42dd-91b5-0ad1cfd3e818 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:37.657258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] The token cache is being invalidate because project 82e6c9d4aab247689623094381fee3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:37.657258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.657746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-df0d6d4b-62cf-42dd-91b5-0ad1cfd3e818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.657855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6cca7a78-7dde-462c-a6ff-a8aabf9247da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:37.662936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 916/1824] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:37 2022] DELETE /identity/v3/projects/82e6c9d4aab247689623094381fee3e3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:37.664440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149a5ff-31ed-45df-baee-866ce9160b11 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.664682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149a5ff-31ed-45df-baee-866ce9160b11 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.664876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7149a5ff-31ed-45df-baee-866ce9160b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.703917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7149a5ff-31ed-45df-baee-866ce9160b11 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:37.708367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7149a5ff-31ed-45df-baee-866ce9160b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.733480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 909/1825] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:37.741736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.742046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.742329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.759978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:37.760188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:37.761240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.770771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 910/1826] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:37.775121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.775809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 917/1827] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:37.783899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.784232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.784516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.786388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0058ffb-d3ce-4220-ae70-ae092ff65bb9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.791148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.791792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.792089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.792389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.793118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:37.793449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.795084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.802877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:37.804066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:37.807237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 911/1828] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:07:37.811648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.812954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 918/1829] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:37.816810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.821095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e1eea-4ba4-44b4-9ab5-29359587242d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.826194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d2311-333d-44d0-b97d-4bf1655f3eab None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.826333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.826871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.827161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.827438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.828179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.830052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.830894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.831528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.831837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.832129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.832852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:37.833163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.834734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.847190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 919/1830] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:07:37.851691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.860215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 912/1831] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:37.861702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07493bb-609b-476a-8e9f-950da239ba84 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.864350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.903205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:37.919306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.919967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.920353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.920633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.921326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.923257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.928683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43edc4eb-5ed7-4032-a215-7c5ae727efa0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:37.933456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:37.934326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:37.934756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:37.935134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:37.935926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:37.937948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:37.948948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 920/1832] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:37.954254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.959047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 913/1833] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:37.962966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:37.972361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029dff7a-55d0-49ad-8a64-96427ca67c79 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.021955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273e0a9-da49-4fa6-97b1-12ef5e16ea4e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.027641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.028681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.029070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.029539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.030253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.032291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.034689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.035361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.035647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.035932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.037866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.040766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.054339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 914/1834] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:37 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:38.063050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 921/1835] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:37 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:38.063224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.064283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.073156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755042c-4d0f-4aad-be24-4de3d87414de admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.076244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642caf49-b8e9-4142-8c5b-94285f48474f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.078655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.079703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.081048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.081522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.082257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.082417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.083213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.083703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.083975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.084744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.085074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.086333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.097302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 915/1836] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:38.098462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 922/1837] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:38.101734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.104127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.113916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b590a-da7a-4fcb-9c0a-c8a9007d59b1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.114555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47321d9b-0ad7-4795-abeb-dcfcfa837218 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.118996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.119972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.120284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.120435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.120744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] PATH_INFO: `/v3/projects/5c70446192ff496da3abc336c2883ffb/users/1cc3bc6b72cf442e8157ff4bdec5d10f/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.120950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.121362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.121654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.123141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.125599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.139478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 923/1838] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:38.140994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.150732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c70446192ff496da3abc336c2883ffb, user_id=1cc3bc6b72cf442e8157ff4bdec5d10f, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.153806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092cfebf-8f95-47ed-9a66-6f64440c4954 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.154476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3079e8f-56ac-44e6-af6a-4f5d93f479e6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.159174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.160102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.160478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.160755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] PATH_INFO: `/v3/projects/f09b899012004716b35d1e8d52822fd1/users/8015eaf62e58463cbb749effe0c3a4bc/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.182127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 916/1839] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:38 2022] PUT /identity/v3/projects/5c70446192ff496da3abc336c2883ffb/users/1cc3bc6b72cf442e8157ff4bdec5d10f/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:38.185259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.185553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.185953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.189471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09b899012004716b35d1e8d52822fd1, user_id=8015eaf62e58463cbb749effe0c3a4bc, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.192629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c872e-3049-45f0-b68c-4eae5855c4d4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.210185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 924/1840] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:38 2022] PUT /identity/v3/projects/f09b899012004716b35d1e8d52822fd1/users/8015eaf62e58463cbb749effe0c3a4bc/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:38.215262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.216164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.216408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.223540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:38.236269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:38.239729 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:38.242346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] MFA Rules not processed for user `1cc3bc6b72cf442e8157ff4bdec5d10f`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:38.243817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:38.331409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 925/1841] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:38.338803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 917/1842] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:38.343349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.344752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.345021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:38.355883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd877c-b027-4dfb-8b3b-73ded39605bd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.361779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.362452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.362744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.363121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.363836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.366618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.385947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:38.420027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 918/1843] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:38 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:38.423038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.456999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8de0d-5c09-41c7-9faa-e5e70b809191 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.536766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae7496-a330-4273-beb7-90ad5a40d457 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.543524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.544370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.544787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.545128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.546291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.548551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.551119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.551877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.552311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.552679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.563371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.564946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.576583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 926/1844] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:07:38.578718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 919/1845] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:38 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:38.582747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.595207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c1ac3-d1ed-423c-9e1c-d9ccd2bec8f4 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.600128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.600771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.601452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.601784 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.602619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.604060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.615254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 927/1846] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:38.619647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.630033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91592d-f436-4dd6-8130-5b6fe0e7bb3a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.635494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.636281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.636597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.637304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] PATH_INFO: `/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded/users/35a667f20c4d48e89976f9edcded4d90/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.656220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b91e512cf541ac9ff52b56a7ae3ded, user_id=35a667f20c4d48e89976f9edcded4d90, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.658854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.675127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 920/1847] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:38 2022] PUT /identity/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded/users/35a667f20c4d48e89976f9edcded4d90/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:38.679105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.692381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40edc4d-5e03-4163-b24b-37167f155595 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.821482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:38.849968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.853734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.854220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.854593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.855901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.858797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0a99a-2559-4d73-8fb5-7b730da44313 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.883014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 928/1848] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:38.888130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:38.907249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea5200-bfe1-489c-b6ae-a361051feab1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:38.914574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:38.915612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.916166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.916575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] PATH_INFO: `/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded/users/35a667f20c4d48e89976f9edcded4d90/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:38.921037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b91e512cf541ac9ff52b56a7ae3ded, user_id=35a667f20c4d48e89976f9edcded4d90, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:38.924836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:38.943093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 921/1849] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:38 2022] PUT /identity/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded/users/35a667f20c4d48e89976f9edcded4d90/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:38.954087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b034a-b175-46af-866c-4f8648b8738d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:38.954651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b034a-b175-46af-866c-4f8648b8738d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:38.955172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b034a-b175-46af-866c-4f8648b8738d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:39.003781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-700b034a-b175-46af-866c-4f8648b8738d None None] MFA Rules not processed for user `35a667f20c4d48e89976f9edcded4d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:39.005331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-700b034a-b175-46af-866c-4f8648b8738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:39.090668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 929/1850] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:39.102903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:39.105266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:39.173624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427dfd1c-567e-4f0d-bb62-78436c29e616 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:39.177632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:39.178419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:39.178804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:39.179320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:39.262603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:39.264928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:39.271107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:39.277761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 922/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:39.710160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:39.710684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:39.711753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:39.777466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] MFA Rules not processed for user `8015eaf62e58463cbb749effe0c3a4bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:39.783357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:39.924410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 930/1852] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:39.928424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.011916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756db6-12b7-4b35-9fd7-fc6af0e6c1df service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.018118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.018689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.018973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.019253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.019949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.021894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.066048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 923/1853] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:39 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:40.070076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.081689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd00f62-5bd2-4e59-acb0-15bae96975c9 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.149752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.150495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.150786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.151066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.151761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.153878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.163857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:40.179809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 931/1854] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:40 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:40.183603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.198665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8fb1e-7fee-4aa0-b6b8-1b5dbbe9d709 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.203921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.207855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.208274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.208463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.209222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.210847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.225116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 924/1855] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:40.229265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.242327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121a7f-c5fc-489d-8faa-8df3d149f274 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.247947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.248884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.249173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.249460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] PATH_INFO: `/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1/users/d296274606bd412a8e517356873f2f90/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.274874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=baacf5e9d52b4ababa4d7f772aa38cb1, user_id=d296274606bd412a8e517356873f2f90, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.277311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.293136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 932/1856] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:40 2022] PUT /identity/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1/users/d296274606bd412a8e517356873f2f90/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:40.297673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.308539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703d1d0-8a8c-4111-88b8-a3d94c5c5210 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.367384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.368049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.368376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.368659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.369417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.371031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a02106-1f04-470b-9dee-ec98bc1f4fa5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.382157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 925/1857] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:40.386390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:40.397265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ee43c-8e65-4c30-9aa7-9222f99197fd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:40.402263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:40.403793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:40.404013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:40.404562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] PATH_INFO: `/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1/users/d296274606bd412a8e517356873f2f90/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:40.406824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=baacf5e9d52b4ababa4d7f772aa38cb1, user_id=d296274606bd412a8e517356873f2f90, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:40.409618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca71397-6b92-4062-aa4f-8091fade8cac admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:40.421350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 933/1858] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:40 2022] PUT /identity/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1/users/d296274606bd412a8e517356873f2f90/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:07:41.405371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:41.405901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:41.406000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:41.408398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 926/1859] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:07:41.423828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:41.424301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:41.424397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:41.455379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:41.456564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:41.603044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 934/1860] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:07:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:41.797987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:41.800530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:41.825250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6501aa6-3ac6-4e56-bae9-a2bb0fef3667 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:41.831979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:41.832670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:41.833012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:41.833338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:41.847848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:41.849585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:41.876474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 927/1861] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:42.069689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:42.071992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:42.151761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb6964-df56-4b28-9f43-9d3e811c5c8d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:42.157578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:42.158592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.158921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.159264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] PATH_INFO: `/v3/users/0be762ddf0cb4d788ed56a5c649477bf` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.160650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be762ddf0cb4d788ed56a5c649477bf)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:42.162704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:42.209026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:42.222789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1362a-e6e5-4068-b0a1-5defcb3bf98f service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:42.232766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:42.234392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.235708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.236536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] PATH_INFO: `/v3/users/1cc3bc6b72cf442e8157ff4bdec5d10f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.238750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cc3bc6b72cf442e8157ff4bdec5d10f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:42.242049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:42.269074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be762ddf0cb4d788ed56a5c649477bf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.278133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:42.297916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be762ddf0cb4d788ed56a5c649477bf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.299273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be762ddf0cb4d788ed56a5c649477bf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.312674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be762ddf0cb4d788ed56a5c649477bf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.338866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ed3f3f45-9fce-468f-9b47-718032188d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be762ddf0cb4d788ed56a5c649477bf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.339716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cc3bc6b72cf442e8157ff4bdec5d10f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.341574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cc3bc6b72cf442e8157ff4bdec5d10f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.343076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cc3bc6b72cf442e8157ff4bdec5d10f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.352739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:42.358276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 935/1862] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:42 2022] DELETE /identity/v3/users/0be762ddf0cb4d788ed56a5c649477bf => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:42.360260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.362466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.365068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cc3bc6b72cf442e8157ff4bdec5d10f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.365373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.396458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4c31b2f1-0281-4ccb-9a16-aec6a5d46461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cc3bc6b72cf442e8157ff4bdec5d10f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:42.409752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:42.411129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:42.416539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 928/1863] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:42 2022] DELETE /identity/v3/users/1cc3bc6b72cf442e8157ff4bdec5d10f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:42.420936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299a2b7-dc29-45fa-89bb-d974160a43a7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.422370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299a2b7-dc29-45fa-89bb-d974160a43a7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.422809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299a2b7-dc29-45fa-89bb-d974160a43a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.459990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7299a2b7-dc29-45fa-89bb-d974160a43a7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:42.462687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7299a2b7-dc29-45fa-89bb-d974160a43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:42.544534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 936/1864] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:42.561726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:42.564959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:42.643812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 929/1865] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:42.652227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.652729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.652922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.664472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0194dbe-a0c4-4748-a64a-1db9d6dd8f6e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:42.669800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:42.670838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.671427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.671860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.684767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:42.686353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:42.699540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 937/1866] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:42.700633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:42.720034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba89d5c-429f-4e70-ac30-a5a870f1972b service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:42.724440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:42.726455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:42.727544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:42.727984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:42.739823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] MFA Rules not processed for user `d296274606bd412a8e517356873f2f90`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:42.741089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:42.742476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:42.743618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:42.754740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 938/1867] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:42.876974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 930/1868] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:42.890891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:42.894070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:43.022721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa57a1d-6ec7-41ee-aa8e-e20aa7cb086a service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:43.029919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:43.031965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:43.032527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:43.033028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:43.035425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:43.050289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:43.129631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:43.131452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:43.145706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 939/1869] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:43.326334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:43.328536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:43.347421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb1d94-feb0-4942-bdcf-7e2b8a87743a None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:43.355696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:43.357657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:43.358420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:43.359350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] PATH_INFO: `/v3/projects/2288c8a2ee544658b0cf357a68c581ea` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:43.361532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=2288c8a2ee544658b0cf357a68c581ea)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:43.364346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:43.438399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2288c8a2ee544658b0cf357a68c581ea', 'request_id': 'req-85084935-e7ca-48ac-af20-fb577f1b8aee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.441909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:43.465464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2288c8a2ee544658b0cf357a68c581ea', 'request_id': 'req-85084935-e7ca-48ac-af20-fb577f1b8aee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.477310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2288c8a2ee544658b0cf357a68c581ea', 'request_id': 'req-85084935-e7ca-48ac-af20-fb577f1b8aee'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.501083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] The token cache is being invalidate because project 2288c8a2ee544658b0cf357a68c581ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:43.501644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.504218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.509394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 931/1870] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:43 2022] DELETE /identity/v3/projects/2288c8a2ee544658b0cf357a68c581ea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:43.511063 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:43.531815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:43.535852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:43.537454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5f21e-d5c3-4486-a2cb-b6eb23f7598a service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:43.677258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:43.679450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:43.679946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:43.680407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] PATH_INFO: `/v3/users/5242d9f4d4694b1ca9c5fed76b473106` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:43.682835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5242d9f4d4694b1ca9c5fed76b473106)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:43.686390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:43.705268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85084935-e7ca-48ac-af20-fb577f1b8aee admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:43.718943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:43.721408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:43.721846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:43.722141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] PATH_INFO: `/v3/projects/5c70446192ff496da3abc336c2883ffb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:43.723565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c70446192ff496da3abc336c2883ffb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:43.733932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:43.800704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5242d9f4d4694b1ca9c5fed76b473106'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.813494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5242d9f4d4694b1ca9c5fed76b473106'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.814646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5242d9f4d4694b1ca9c5fed76b473106'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.827909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5242d9f4d4694b1ca9c5fed76b473106'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.839617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c70446192ff496da3abc336c2883ffb', 'request_id': 'req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.848140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c70446192ff496da3abc336c2883ffb', 'request_id': 'req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.852665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5242d9f4d4694b1ca9c5fed76b473106'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.856556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c70446192ff496da3abc336c2883ffb', 'request_id': 'req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.865995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 940/1871] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:43 2022] DELETE /identity/v3/users/5242d9f4d4694b1ca9c5fed76b473106 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:43.900069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] The token cache is being invalidate because project 5c70446192ff496da3abc336c2883ffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:43.902289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.904032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:43.909195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 932/1872] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:43 2022] DELETE /identity/v3/projects/5c70446192ff496da3abc336c2883ffb => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:43.910360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:43.913510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:43.975198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:44.015745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83f0ee-c223-482f-9f8a-2304687d8b10 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:44.025772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:44.026990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.027359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.027744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] PATH_INFO: `/v3/users/35a667f20c4d48e89976f9edcded4d90` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.031214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35a667f20c4d48e89976f9edcded4d90)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:44.034907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:44.119084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35a667f20c4d48e89976f9edcded4d90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.134591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35a667f20c4d48e89976f9edcded4d90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.135372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35a667f20c4d48e89976f9edcded4d90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.157913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35a667f20c4d48e89976f9edcded4d90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.182626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d4bedc09-6239-430d-9933-44a3b036621d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35a667f20c4d48e89976f9edcded4d90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.194188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 941/1873] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:07:43 2022] DELETE /identity/v3/users/35a667f20c4d48e89976f9edcded4d90 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:44.448949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:44.451567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:44.551510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016f2bb-c1c9-434b-8ef9-8bf51111ecc2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:44.569748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:44.635914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:44.636571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.636958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.637375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] PATH_INFO: `/v3/projects/2df213a09db341648bf078bcfffc639a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.638698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df213a09db341648bf078bcfffc639a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:44.640818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:44.695682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df213a09db341648bf078bcfffc639a', 'request_id': 'req-2dbf5ae2-b83e-407d-b331-03388daf9768'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.701788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df213a09db341648bf078bcfffc639a', 'request_id': 'req-2dbf5ae2-b83e-407d-b331-03388daf9768'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.711200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df213a09db341648bf078bcfffc639a', 'request_id': 'req-2dbf5ae2-b83e-407d-b331-03388daf9768'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.732065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] The token cache is being invalidate because project 2df213a09db341648bf078bcfffc639a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:44.732608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.733240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.739878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.740808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 933/1874] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:44 2022] DELETE /identity/v3/projects/2df213a09db341648bf078bcfffc639a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:44.741084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.741506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.741713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:44.743415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:44.773117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:44.777292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:44.837255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf5ae2-b83e-407d-b331-03388daf9768 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:44.843470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:44.844449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.844798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.851318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 942/1875] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:44.851464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] PATH_INFO: `/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.851576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:44.851576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:44.851918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b91e512cf541ac9ff52b56a7ae3ded)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:44.854275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:44.906597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:44.907685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25fdae0-1052-4367-be25-4a3cd6710616 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:44.908463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b91e512cf541ac9ff52b56a7ae3ded', 'request_id': 'req-94f9707e-c88e-4418-bc33-9e1a2e98bd99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.912667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:44.918017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b91e512cf541ac9ff52b56a7ae3ded', 'request_id': 'req-94f9707e-c88e-4418-bc33-9e1a2e98bd99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.928096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b91e512cf541ac9ff52b56a7ae3ded', 'request_id': 'req-94f9707e-c88e-4418-bc33-9e1a2e98bd99'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.940787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] The token cache is being invalidate because project 49b91e512cf541ac9ff52b56a7ae3ded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:07:44.941084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.941718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-94f9707e-c88e-4418-bc33-9e1a2e98bd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:07:44.943538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 934/1876] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:07:44 2022] DELETE /identity/v3/projects/49b91e512cf541ac9ff52b56a7ae3ded => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:44.944564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab087-9250-4299-8ccc-4329d02f556c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.944873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab087-9250-4299-8ccc-4329d02f556c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.945151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab087-9250-4299-8ccc-4329d02f556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.966054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-cf0ab087-9250-4299-8ccc-4329d02f556c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:44.967195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cf0ab087-9250-4299-8ccc-4329d02f556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:44.967494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:44.968186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:44.968480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:44.968765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:44.970650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:45.032164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 935/1877] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:45.033459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c077-2da9-4db2-b2b5-cccf02e5ba05 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.033752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c077-2da9-4db2-b2b5-cccf02e5ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.034035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3c077-2da9-4db2-b2b5-cccf02e5ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.034401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.035933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.046643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 943/1878] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:45.047305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.051968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-64a3c077-2da9-4db2-b2b5-cccf02e5ba05 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:45.053206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-64a3c077-2da9-4db2-b2b5-cccf02e5ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:45.057027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14f6d7-e665-4bc9-a40d-bdaebcab84ba service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.062138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.069082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.069423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 936/1879] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:45.069531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.069531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.069531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:45.069531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.069531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.071046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.071405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.071756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.078205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 944/1880] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:45.082877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.095030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:45.096059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733478f-2ed3-4eec-a797-ed03b608ef1e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.096288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:45.101244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.102073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.102398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.102680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.103385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.105408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.110804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 937/1881] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:07:45.111923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.122936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f04ab-99a8-491a-a2af-686d489c008d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.130285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.130950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.131233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.131506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.132288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:07:45.132592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.134307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.139136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 945/1882] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:45.139838 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.147226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 938/1883] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:07:45.152546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.163533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54224c9-9ed3-4de8-b765-506da92ea015 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.217480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bdb4-0da9-4ee3-86f8-657752cebb64 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.223296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.224254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.224620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.224961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.225766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.227892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.259152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 946/1884] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:45.260054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.273406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4dc2b-af4e-484d-a2d3-fe7ff2af7697 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.282225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.283383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.283672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.284047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.284921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.286960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.293575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:45.306772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 947/1885] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:07:45.307925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.310753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.311645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.312097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.312706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.313609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.317320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.324418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099f3cb-61f3-423e-bfa1-e8c48f8a9d10 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.330698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.332041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.332668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.333060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.333864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.336878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.357822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 948/1886] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:45.358826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 939/1887] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:45.359010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.360923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.375022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f752c-3a2f-49ae-b772-22e8ad01caa8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.432293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:45.467808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64220-ba82-41ba-b514-4cb6f92d6aa0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.473426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.474681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.474883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.476134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] PATH_INFO: `/v3/projects/93d1bdfee838452da48a80854e694447/users/c25c5804072142418b52b0a412820d15/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.478325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.480574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.481907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.482794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.483518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.487940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.512921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d1bdfee838452da48a80854e694447, user_id=c25c5804072142418b52b0a412820d15, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.521062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1c192-2f82-4df1-a435-21dee52f82eb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.528966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 949/1888] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:45.530749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.553058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 940/1889] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:45 2022] PUT /identity/v3/projects/93d1bdfee838452da48a80854e694447/users/c25c5804072142418b52b0a412820d15/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:45.563332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decf542-0c75-4bdb-9a8d-d219720a2c4a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.565627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.565993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.566349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.599486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] MFA Rules not processed for user `c25c5804072142418b52b0a412820d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:45.601076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:45.672173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.673804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.674224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.674662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.675549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.681937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.696479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 941/1890] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:45.699048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.711178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 950/1891] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:45.717591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.725487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becd00a-0309-4ef6-919a-39f5b33fa66c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.732032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.732207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54f5ee-5829-46ce-8099-44bf0962d2a6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.733109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.733633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.734269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.735178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.737363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.737477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.738030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.738369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.738652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.740058 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.742242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.763179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 951/1892] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:45.767688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.779530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e680765-8d97-4f52-8ab3-70dbfdaf55b9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.780784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 942/1893] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:07:45.785219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:45.865157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:45.866284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.866412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.866686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] PATH_INFO: `/v3/projects/85476dbcd1f84580a6fbb1c76f1881b2/users/d3729ae8e9f7417b85d33dfabedc8c1a/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.877870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:45.892947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85476dbcd1f84580a6fbb1c76f1881b2, user_id=d3729ae8e9f7417b85d33dfabedc8c1a, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:45.898825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262c34-5090-4ad8-ae1a-b4f1cb4ad643 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:45.928453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 952/1894] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:45 2022] PUT /identity/v3/projects/85476dbcd1f84580a6fbb1c76f1881b2/users/d3729ae8e9f7417b85d33dfabedc8c1a/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:45.939790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:45.940191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:45.940623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:45.943072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a98c-ce48-47d2-acfc-fa1e8718fac6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:45.982847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:45.994029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] MFA Rules not processed for user `d3729ae8e9f7417b85d33dfabedc8c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:45.997910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:46.051575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.052301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.052590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.052874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.053715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.055596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.085622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 943/1895] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/users => generated 294 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:07:46.090388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.113537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ca61-1249-44c7-ad65-d09753c8ffa3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:46.125647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.126812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.127098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.127386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.128152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.130483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.156030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 944/1896] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:46.161132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.167266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 953/1897] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:46.186935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.188481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe47851-b37c-4617-92d4-5957cae38929 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:46.190127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:46.197051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.198019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.198311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.200141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] PATH_INFO: `/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43/users/5f1b1c3ac844457b906af5cffef3e307/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.246572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19f99bbc0954555bfa74c3c89bb5c43, user_id=5f1b1c3ac844457b906af5cffef3e307, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.251064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.274999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 945/1898] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:46 2022] PUT /identity/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43/users/5f1b1c3ac844457b906af5cffef3e307/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:46.280195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.306987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12911f03-5c9a-4c9a-963e-fceeb695498a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:46.375954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.376751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.377342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.377810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.379484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2717f4-243c-4e9d-a168-ba0cab0052d4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:46.386252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.388867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.389200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.389485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.390601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.393916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.410807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 946/1899] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:07:46.411735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.422976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed82c7-9167-448c-b749-cc0e3102d6df admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:46.438685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:46.439605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.439920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.440168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] PATH_INFO: `/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43/users/5f1b1c3ac844457b906af5cffef3e307/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.442638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19f99bbc0954555bfa74c3c89bb5c43, user_id=5f1b1c3ac844457b906af5cffef3e307, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.445705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a62d37-e3ef-434a-bfd7-122ee8bfddb7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.471940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:46.480514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 947/1900] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:07:46 2022] PUT /identity/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43/users/5f1b1c3ac844457b906af5cffef3e307/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:07:46.487514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:46.490229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:46.491056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:46.491793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:46.491793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:46.503136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 954/1901] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:46.543176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] MFA Rules not processed for user `5f1b1c3ac844457b906af5cffef3e307`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:07:46.547813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:46.771468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:07:46.798238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 948/1902] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:07:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:07:46.859965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:46.862392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:07:46.946291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ac8b3-d0e6-43f3-9d4a-edecf357585c service glance] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:47.026073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:47.026918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:47.027298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:47.027592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:47.097753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:47.099382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c58e0-3732-4f20-88b4-09f119646b53 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:47.110972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 955/1903] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:53.088664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:07:53.098203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e2a2b-56a5-4bcb-86d8-7dfe89a0a6ab None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:07:53.102046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:07:53.102714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:07:53.103012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:07:53.103301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:07:53.113657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:07:53.115324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a251c9e-fd73-4aa6-9fe1-b85306afc032 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:07:53.126092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 949/1904] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:07:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:07:57.104911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=74 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:03.120645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=8 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:05.958981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:05.959545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:05.959545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:05.962494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 956/1905] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:08:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:08:05.973729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bbd5-6c50-4e26-aff0-4603eee1580b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:05.974229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bbd5-6c50-4e26-aff0-4603eee1580b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:05.974559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242bbd5-6c50-4e26-aff0-4603eee1580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:05.996156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2242bbd5-6c50-4e26-aff0-4603eee1580b None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:05.997627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2242bbd5-6c50-4e26-aff0-4603eee1580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:06.109402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 950/1906] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:08:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:06.123198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:06.125024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:06.146536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d0dca7-436e-41a4-a68e-db542ccca42c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:06.166825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:06.167481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:06.167481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:06.168367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:06.179635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:06.181634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e58c47-0e34-41c6-96dc-167541314ba5 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:06.193315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 957/1907] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:06.736953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10392f24-7b2c-432a-9007-81fba047e33c None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:06.737248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10392f24-7b2c-432a-9007-81fba047e33c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:06.737528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10392f24-7b2c-432a-9007-81fba047e33c None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:06.739006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 951/1908] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:08:06.744634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:06.744863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:06.745240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:06.764408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] MFA Rules not processed for user `2aaa83fc8b334b07b67a2ff908a4a105`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:06.765612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:06.774796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 958/1909] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:08:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:06.800450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10392f24-7b2c-432a-9007-81fba047e33c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:06.826887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:06.828407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:06.831845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10392f24-7b2c-432a-9007-81fba047e33c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:06.849136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:06.850188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:06.850582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:06.851293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:06.860418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:06.862138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:06.872996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 952/1910] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:06.905706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427ffd5-db60-4c62-8fb4-1330a9362598 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:06.910340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:06.911210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:06.911494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:06.911767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] PATH_INFO: `/v3/users/8015eaf62e58463cbb749effe0c3a4bc` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:06.913011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8015eaf62e58463cbb749effe0c3a4bc)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:06.914946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:06.963010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8015eaf62e58463cbb749effe0c3a4bc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:06.969756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8015eaf62e58463cbb749effe0c3a4bc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:06.970428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8015eaf62e58463cbb749effe0c3a4bc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:06.977653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8015eaf62e58463cbb749effe0c3a4bc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:06.989980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8015eaf62e58463cbb749effe0c3a4bc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:06.995933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 959/1911] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:06 2022] DELETE /identity/v3/users/8015eaf62e58463cbb749effe0c3a4bc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:07.173267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:07.196513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5de89-124c-4bee-81ea-260934bc495a service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:07.263619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:07.264587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:07.264870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:07.265147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] PATH_INFO: `/v3/projects/f09b899012004716b35d1e8d52822fd1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:07.266346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f09b899012004716b35d1e8d52822fd1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:07.268542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:07.315994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f09b899012004716b35d1e8d52822fd1', 'request_id': 'req-7749099f-c950-4f8f-9df7-7b369bf46d1e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.321613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f09b899012004716b35d1e8d52822fd1', 'request_id': 'req-7749099f-c950-4f8f-9df7-7b369bf46d1e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.327812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f09b899012004716b35d1e8d52822fd1', 'request_id': 'req-7749099f-c950-4f8f-9df7-7b369bf46d1e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.340278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] The token cache is being invalidate because project f09b899012004716b35d1e8d52822fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:07.340567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.341694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.343269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 953/1912] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:07 2022] DELETE /identity/v3/projects/f09b899012004716b35d1e8d52822fd1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:07.347157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:07.348740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:07.361910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd01016-28b2-43c6-865c-10222219f653 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:07.420091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:07.420981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:07.421266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:07.421545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] PATH_INFO: `/v3/users/d296274606bd412a8e517356873f2f90` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:07.422724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d296274606bd412a8e517356873f2f90)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:07.424675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:07.426129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:07.475055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd296274606bd412a8e517356873f2f90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.484119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd296274606bd412a8e517356873f2f90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.484810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd296274606bd412a8e517356873f2f90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.492308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd296274606bd412a8e517356873f2f90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.504529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a8530f23-733b-4b91-ab0a-dffc815b668e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd296274606bd412a8e517356873f2f90'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.510450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 960/1913] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:07 2022] DELETE /identity/v3/users/d296274606bd412a8e517356873f2f90 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:07.688097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:07.689758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:07.703426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749099f-c950-4f8f-9df7-7b369bf46d1e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:07.717204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:07.769701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:07.770566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:07.770860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:07.771137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] PATH_INFO: `/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:07.772354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=baacf5e9d52b4ababa4d7f772aa38cb1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:07.774249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:07.819173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baacf5e9d52b4ababa4d7f772aa38cb1', 'request_id': 'req-19ce38e1-5a29-4bfc-bde1-831337443d6b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.829015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baacf5e9d52b4ababa4d7f772aa38cb1', 'request_id': 'req-19ce38e1-5a29-4bfc-bde1-831337443d6b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.835968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baacf5e9d52b4ababa4d7f772aa38cb1', 'request_id': 'req-19ce38e1-5a29-4bfc-bde1-831337443d6b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.851700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] The token cache is being invalidate because project baacf5e9d52b4ababa4d7f772aa38cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:07.851994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.852673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-19ce38e1-5a29-4bfc-bde1-831337443d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:07.854257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 954/1914] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:07 2022] DELETE /identity/v3/projects/baacf5e9d52b4ababa4d7f772aa38cb1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:07.859900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:07.860239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:07.860541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:07.878733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:07.880048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:07.946897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 961/1915] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:07.956196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:07.956496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:07.956784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:07.975775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:07.977095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:07.985131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 955/1916] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:07.989090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:07.998755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf8a86-ad45-4470-b15a-aa32b488051f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.003772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.004481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.004768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.005049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.005811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:08.006145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.007739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.020518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 962/1917] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:08.024673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.035130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6b247-8821-4505-adf0-4e19e0c68cf5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.039777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.040464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.040749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.041027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.041767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.043653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.068358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 956/1918] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:08 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:08.072696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.083327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdac29-75b7-45f5-b9e5-81d37503f265 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.142334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.143003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.143287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.143565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.144293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.146205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.155495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:08.167889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 963/1919] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:08 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:08.171541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.181841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728da8-e495-497d-855d-f595c026219c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.186932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.187586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.187873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.188179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.188901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.190489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.203291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 957/1920] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:08.207187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.217130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bb3d8-2e64-441f-9a58-193e3051a7f8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.221701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.222351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.222636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.223235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.223616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.225233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61365750-fb7c-4c65-b83f-9b670577764a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.238356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 964/1921] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:08:08.239525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.250206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0b86a-d305-4d7e-8b42-c670187271ae admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.255644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.256553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.256867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.257147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] PATH_INFO: `/v3/projects/2e36d544e3214d669123b6a900c54f64/users/5dbd155457cc48bb99a2407c52288d8c/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.277077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e36d544e3214d669123b6a900c54f64, user_id=5dbd155457cc48bb99a2407c52288d8c, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.279745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.294759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 958/1922] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:08 2022] PUT /identity/v3/projects/2e36d544e3214d669123b6a900c54f64/users/5dbd155457cc48bb99a2407c52288d8c/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:08.307658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.307989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.308304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.336163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] MFA Rules not processed for user `5dbd155457cc48bb99a2407c52288d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:08.337368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:08.402268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 965/1923] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:08:08.412980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:08.414865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:08.458846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:08.474363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31691e-a6f0-4fd1-86e5-d79fe5c3a399 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:08.478294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:08.478956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:08.479226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:08.479519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:08.489116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:08.490596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8865d6-08ee-40fc-add7-b9aa54ccf7f7 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:08.500192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 959/1924] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:18.398760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=39 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:18.495430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=8 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:21.020375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:21.129476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e144127-9a45-4a8a-8554-81158a482778 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:21.135961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:21.137198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.137695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.138772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] PATH_INFO: `/v3/users/5dbd155457cc48bb99a2407c52288d8c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.141313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dbd155457cc48bb99a2407c52288d8c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:21.149965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:21.227771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dbd155457cc48bb99a2407c52288d8c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.237630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dbd155457cc48bb99a2407c52288d8c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.238443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dbd155457cc48bb99a2407c52288d8c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.250157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dbd155457cc48bb99a2407c52288d8c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.270461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dbd155457cc48bb99a2407c52288d8c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.279075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 966/1925] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:21 2022] DELETE /identity/v3/users/5dbd155457cc48bb99a2407c52288d8c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:21.284091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.284579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.284868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.304670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:21.305981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:21.366907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 960/1926] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:08:21.376643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:21.378144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:21.437138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f52aea-92de-4574-a08d-f6c2c9d214a6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:21.440816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:21.441461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.441743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.442018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.450946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:21.452449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5557f-0625-4a8d-a4c5-7714da90992a service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:21.461675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 967/1927] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:21.731297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:21.733477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:21.750034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3308b-10d0-43cf-a308-b4bb7ba2decb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:21.755604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:21.756658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.756975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.757239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] PATH_INFO: `/v3/projects/2e36d544e3214d669123b6a900c54f64` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.758583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e36d544e3214d669123b6a900c54f64)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:21.760577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:21.809937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e36d544e3214d669123b6a900c54f64', 'request_id': 'req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.816216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e36d544e3214d669123b6a900c54f64', 'request_id': 'req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.823216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e36d544e3214d669123b6a900c54f64', 'request_id': 'req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.837811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] The token cache is being invalidate because project 2e36d544e3214d669123b6a900c54f64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:21.838105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.838732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ea12e7e9-4ae4-4d93-b08f-0982a1b81164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:21.840363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 961/1928] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:21 2022] DELETE /identity/v3/projects/2e36d544e3214d669123b6a900c54f64 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:21.845283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.845667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.846017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.863465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:21.864819 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:21.882340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:21.930613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 968/1929] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:21.938809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.939103 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.939395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.957415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:21.958616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:21.966664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 962/1930] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:21.971025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:21.980805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced3c85-8899-4e3d-893c-57764e9e057c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:21.985859 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:21.986506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:21.986790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:21.987066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:21.987789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:21.988094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:21.989715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.001528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 969/1931] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:08:22.005698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.015475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf7536-5028-4bbc-8465-297e6520c442 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.020392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.021062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.021351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.021631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.022319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.024258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.049874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 963/1932] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:22 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:22.053969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.063952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c8003-7d33-4765-83c7-f405f4b9d87e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.119660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.120509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.120794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.121070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.121787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.123676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.143387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 970/1933] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:22 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:22.147501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.158580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1c810-9316-4ce3-be3e-21d6ce6297bd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.163147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.163800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.164087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.164396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.165120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.166744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.177030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 964/1934] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:22.180455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.190161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb6f5f-4d99-4401-864b-a7ce181635a3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.194587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.195220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.195527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.195803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.196576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.198150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b348c2a-7bfe-4ac1-a8f7-52988c8a53fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.208243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 971/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:22.212038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.222471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e482a-de2f-4778-8a4b-c04060172aae admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.227127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.228000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.228319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.228604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] PATH_INFO: `/v3/projects/20a5d413451c4fe79fb8a521ddf29326/users/fc02ed62c23443ab9f968a71c2c674bf/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.245083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a5d413451c4fe79fb8a521ddf29326, user_id=fc02ed62c23443ab9f968a71c2c674bf, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.247787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.260836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 965/1936] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:22 2022] PUT /identity/v3/projects/20a5d413451c4fe79fb8a521ddf29326/users/fc02ed62c23443ab9f968a71c2c674bf/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:22.272578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.272936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.273224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.302650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] MFA Rules not processed for user `fc02ed62c23443ab9f968a71c2c674bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:22.303855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:22.391237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 972/1937] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:22.443779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:22.445728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:22.470512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:22.513098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480221f7-0c05-415f-b6a6-dd999c988df9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:22.519169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:22.520450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:22.520450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:22.520450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:22.530361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:22.531790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4affb-fecd-4074-ae20-6317caac6c10 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:22.543645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 966/1938] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:23.462779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:23.464212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:23.485297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:23.532964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2762d-daa2-4daa-869c-0a0db61523a8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:23.537405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:23.538269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:23.538554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:23.538831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] PATH_INFO: `/v3/users/c25c5804072142418b52b0a412820d15` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:23.540042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25c5804072142418b52b0a412820d15)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:23.541982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:23.585421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25c5804072142418b52b0a412820d15'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:23.592693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25c5804072142418b52b0a412820d15'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:23.593350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25c5804072142418b52b0a412820d15'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:23.600444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25c5804072142418b52b0a412820d15'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:23.612502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25c5804072142418b52b0a412820d15'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:23.618235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 973/1939] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:23 2022] DELETE /identity/v3/users/c25c5804072142418b52b0a412820d15 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:23.622664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefaea0-25c7-4b25-aec5-6e29b085df00 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:23.623145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefaea0-25c7-4b25-aec5-6e29b085df00 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:23.623257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefaea0-25c7-4b25-aec5-6e29b085df00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:23.640752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ceefaea0-25c7-4b25-aec5-6e29b085df00 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:23.642230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ceefaea0-25c7-4b25-aec5-6e29b085df00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:23.715096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 967/1940] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:23.739304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:23.740981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:23.812079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd13651-3790-43f7-b1ec-c473586cc354 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:23.816221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:23.817042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:23.817326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:23.817654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:23.828383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:23.832148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:23.846043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 974/1941] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:23.952234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:23.952575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:23.952923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:23.971888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] MFA Rules not processed for user `fc02ed62c23443ab9f968a71c2c674bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:23.973252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:24.043020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 968/1942] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:24.146334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:24.147755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:24.168076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6738bc-822a-410e-8d4b-1c4fb2c1bc22 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:24.175304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:24.176574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:24.177017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:24.177405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] PATH_INFO: `/v3/projects/93d1bdfee838452da48a80854e694447` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:24.179011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d1bdfee838452da48a80854e694447)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:24.181165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:24.223117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:24.234448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d1bdfee838452da48a80854e694447', 'request_id': 'req-8bce6edf-3ba0-4e17-8ae1-712bce664377'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.246393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d1bdfee838452da48a80854e694447', 'request_id': 'req-8bce6edf-3ba0-4e17-8ae1-712bce664377'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.253060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d1bdfee838452da48a80854e694447', 'request_id': 'req-8bce6edf-3ba0-4e17-8ae1-712bce664377'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.265951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] The token cache is being invalidate because project 93d1bdfee838452da48a80854e694447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:24.266248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.267167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.268763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 975/1943] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:24 2022] DELETE /identity/v3/projects/93d1bdfee838452da48a80854e694447 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:24.272914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:24.287719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7737101-5d03-48a5-a92f-af5a551ce811 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:24.351832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:24.352965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:24.353461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:24.353749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] PATH_INFO: `/v3/users/5f1b1c3ac844457b906af5cffef3e307` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:24.355003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f1b1c3ac844457b906af5cffef3e307)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:24.357001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:24.369020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:24.410485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f1b1c3ac844457b906af5cffef3e307'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.411145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f1b1c3ac844457b906af5cffef3e307'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.411767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f1b1c3ac844457b906af5cffef3e307'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.419925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f1b1c3ac844457b906af5cffef3e307'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.432921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-03c86b5e-dfb1-4c52-992b-bf7195118c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f1b1c3ac844457b906af5cffef3e307'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.446995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 969/1944] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:24 2022] DELETE /identity/v3/users/5f1b1c3ac844457b906af5cffef3e307 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:24.738475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:24.740020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:24.754679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6edf-3ba0-4e17-8ae1-712bce664377 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:24.824150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:24.825166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:24.825565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:24.825938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] PATH_INFO: `/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:24.827294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19f99bbc0954555bfa74c3c89bb5c43)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:24.829878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:24.889594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19f99bbc0954555bfa74c3c89bb5c43', 'request_id': 'req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.899124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19f99bbc0954555bfa74c3c89bb5c43', 'request_id': 'req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.908422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19f99bbc0954555bfa74c3c89bb5c43', 'request_id': 'req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.927592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] The token cache is being invalidate because project a19f99bbc0954555bfa74c3c89bb5c43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:24.927888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.929248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9b145a69-58ca-4a02-a0e9-9ee7f24f2eed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:24.936060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 976/1945] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:24 2022] DELETE /identity/v3/projects/a19f99bbc0954555bfa74c3c89bb5c43 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:24.940628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:24.941429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:24.941865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:24.966300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:24.968367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:25.036296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 970/1946] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:25.044047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.044350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.044636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.063297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:25.064550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:25.074824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 977/1947] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:25.079600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.092790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c74bf8-effc-46fc-a96a-e24c25366f20 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.097774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.098424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.098712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.098991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.099757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:25.100069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.101785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.113727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 971/1948] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:25.118091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.127837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf22205-0d23-4884-bd86-0b4a57e04b2a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.132726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.133470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.133877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.134540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.134935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.136871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.170314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 978/1949] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:08:25.171676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.184554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5b45-3a97-4b3d-8f41-24f39ffb9a9f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.243010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.243674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.243958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.244271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.244952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.246848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.266310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 972/1950] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:25.270817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.281718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1888-d303-4adb-bffd-641effcb3df9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.286322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.286979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.287287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.287552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.288308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.289908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.300379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 979/1951] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:25.304089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.310305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:25.315287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe16de-3bc5-499b-bdb9-84e9beb8f60d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.321618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.322346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.322795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.323149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.324159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.325831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0d72e-7346-4c5c-9f17-32fbbf6b3214 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.337176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 973/1952] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:25.341288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.351152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17529627-5f4e-46e2-b1c8-d50641d6b612 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.355713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.356623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.356907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.357183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] PATH_INFO: `/v3/projects/6a3ab0b0885e483e800ef6ff9999514b/users/f8f93a037ca64f0fa3b9b9d44a46fa4b/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.369042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:25.374473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a3ab0b0885e483e800ef6ff9999514b, user_id=f8f93a037ca64f0fa3b9b9d44a46fa4b, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.377482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.392380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 980/1953] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:25 2022] PUT /identity/v3/projects/6a3ab0b0885e483e800ef6ff9999514b/users/f8f93a037ca64f0fa3b9b9d44a46fa4b/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:25.402485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.402794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.403070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.429169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] MFA Rules not processed for user `f8f93a037ca64f0fa3b9b9d44a46fa4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:25.430661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:25.493601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 974/1954] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:08:25.497972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.561160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ac314-f52d-4f6d-bb68-adc999bfc108 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.566012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.566662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.566946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.567267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.567967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.570063 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.594421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 981/1955] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:08:25.598452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.609081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61957660-663a-4719-ba3b-5c92c8622b4f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.665805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.666475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.666766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.667048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.667753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.669713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.693074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 975/1956] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:08:25.694956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.706352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd716-58bf-4020-a66d-afede5d326a4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.711165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.711830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.712132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.712426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.713158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.714756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.726254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 982/1957] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:25.732171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.746560 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568f2ad-4ab5-4834-aeb9-bd64c6e880e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.751392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.752312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.752720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.753010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] PATH_INFO: `/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0/users/b486a55b2d7d46bf91aed5907222523b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.768961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa51aac9ce54be892c1c862a75ca5e0, user_id=b486a55b2d7d46bf91aed5907222523b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.771641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.785579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 976/1958] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:25 2022] PUT /identity/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0/users/b486a55b2d7d46bf91aed5907222523b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:25.789549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.800015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709810e-3b61-4e93-aea9-53ffb4b2dbc5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.859760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.860456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.860734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.861035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.861804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.863524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc45a9d-2922-4b32-a685-aaca2884eb9b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.874080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 983/1959] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:08:25.878183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:25.888541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e0205-ed42-419e-b4d4-846d8be5065b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:25.893469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:25.894347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.894633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.894909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] PATH_INFO: `/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0/users/b486a55b2d7d46bf91aed5907222523b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.896951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa51aac9ce54be892c1c862a75ca5e0, user_id=b486a55b2d7d46bf91aed5907222523b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:25.899603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:25.910337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 977/1960] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:25 2022] PUT /identity/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0/users/b486a55b2d7d46bf91aed5907222523b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:25.921924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:25.922217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:25.922525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:25.951533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] MFA Rules not processed for user `b486a55b2d7d46bf91aed5907222523b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:25.951655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:25.999123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:26.021455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 984/1961] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:26.033187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:26.034957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:26.074823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:26.095040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dd93eb-79d9-4e88-8cf7-e02e48bfdc0e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:26.098709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:26.099381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.099667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.099948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.110173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:26.111600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6389d6-7f1b-4c1a-88de-b231dadc8c67 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:26.121313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 978/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:26.150285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:26.212287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55b21d-b641-443f-9d86-91c62e3c95a8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:26.216815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:26.217674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.217958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.218237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] PATH_INFO: `/v3/users/f8f93a037ca64f0fa3b9b9d44a46fa4b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.219430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f93a037ca64f0fa3b9b9d44a46fa4b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:26.221374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:26.265720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f93a037ca64f0fa3b9b9d44a46fa4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.272649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f93a037ca64f0fa3b9b9d44a46fa4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.273323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f93a037ca64f0fa3b9b9d44a46fa4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.280202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f93a037ca64f0fa3b9b9d44a46fa4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.291666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f93a037ca64f0fa3b9b9d44a46fa4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.297685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 985/1963] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:26 2022] DELETE /identity/v3/users/f8f93a037ca64f0fa3b9b9d44a46fa4b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:26.302131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.302421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.302701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.320338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:26.321538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:26.384471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 979/1964] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:26.394213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:26.395629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:26.450911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e86203-da36-4af0-bfa4-b020693ca3fe admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:26.455118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:26.455749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.456028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.456323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.465111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:26.466526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:26.476333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 986/1965] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:26.745649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:26.747076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:26.764490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba55590-3c68-4895-ae73-793eb75e551c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:26.770394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:26.771442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.771836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.772264 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] PATH_INFO: `/v3/projects/6a3ab0b0885e483e800ef6ff9999514b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.773642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a3ab0b0885e483e800ef6ff9999514b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:26.775549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:26.822557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a3ab0b0885e483e800ef6ff9999514b', 'request_id': 'req-532f5260-a55f-4526-a917-4fc65ad504e8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.831400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a3ab0b0885e483e800ef6ff9999514b', 'request_id': 'req-532f5260-a55f-4526-a917-4fc65ad504e8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.837861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a3ab0b0885e483e800ef6ff9999514b', 'request_id': 'req-532f5260-a55f-4526-a917-4fc65ad504e8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.850877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] The token cache is being invalidate because project 6a3ab0b0885e483e800ef6ff9999514b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:26.851165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.851786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.853574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 980/1966] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:26 2022] DELETE /identity/v3/projects/6a3ab0b0885e483e800ef6ff9999514b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:26.857344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:26.867094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d78b7-faea-4f11-b2d6-d8ad786dd131 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:26.870914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:26.926988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:26.927862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:26.928177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:26.928465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] PATH_INFO: `/v3/users/b486a55b2d7d46bf91aed5907222523b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:26.929652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b486a55b2d7d46bf91aed5907222523b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:26.931595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:26.987001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b486a55b2d7d46bf91aed5907222523b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.999286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b486a55b2d7d46bf91aed5907222523b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:26.999929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b486a55b2d7d46bf91aed5907222523b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.013295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b486a55b2d7d46bf91aed5907222523b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.026995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d5104f32-3cb2-4a9c-b8aa-febcac418ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b486a55b2d7d46bf91aed5907222523b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.033591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 987/1967] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:26 2022] DELETE /identity/v3/users/b486a55b2d7d46bf91aed5907222523b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:27.377410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.378973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:27.398677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f5260-a55f-4526-a917-4fc65ad504e8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.458022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.458794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.459074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.459351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] PATH_INFO: `/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.460582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa51aac9ce54be892c1c862a75ca5e0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.462498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.464762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:27.513720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa51aac9ce54be892c1c862a75ca5e0', 'request_id': 'req-e890436e-264d-4148-a894-2acce86fdb39'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.519491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa51aac9ce54be892c1c862a75ca5e0', 'request_id': 'req-e890436e-264d-4148-a894-2acce86fdb39'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.527501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa51aac9ce54be892c1c862a75ca5e0', 'request_id': 'req-e890436e-264d-4148-a894-2acce86fdb39'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.539646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] The token cache is being invalidate because project aaa51aac9ce54be892c1c862a75ca5e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:27.539941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.540614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e890436e-264d-4148-a894-2acce86fdb39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:27.542156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 981/1968] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:27 2022] DELETE /identity/v3/projects/aaa51aac9ce54be892c1c862a75ca5e0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:27.547670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.547962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.548272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.566485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:27.567676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:27.627224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 988/1969] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:27.635305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.635598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.635884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.654892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:27.656298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:27.669123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 982/1970] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:27.673953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.683946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83274c-a700-424c-8813-6b16e1199269 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.688798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.689455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.689740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.690019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.690747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:27.691055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.692822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.708131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 989/1971] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:27.712402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.722148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2691f-f5ba-4382-8e8a-ee500fc0a227 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.726757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.727403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.727691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.727970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.728712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.730618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.756156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 983/1972] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:27 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:27.760209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.770419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e12a1e-0ce6-41cf-9a3b-3dfa083dc6f6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.779030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:27.826627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.827286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.827571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.827850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.828583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.830538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.849777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 990/1973] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:27 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:27.853833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.864539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e018dc3-21f2-47d0-aa53-b533e5abe50d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.869073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.869729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.870016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.870296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.871024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.872655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.883322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 984/1974] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:27.886975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.897978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90052cd7-db28-48ed-bd6b-ee05d981672a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.902730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.903382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.903667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.903945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.904771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.906443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030cca70-ecde-4d16-8e4a-0d6ca5458cc7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.916320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 991/1975] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:27.920905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:27.930704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91186c6f-137a-48fa-9ccf-1a7678a8547a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:27.935223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:27.936951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.937180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.937454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] PATH_INFO: `/v3/projects/7940b6ff003d4ba89e7b65425996eede/users/357e127d423e43fea29162663ecd8666/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:27.951728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7940b6ff003d4ba89e7b65425996eede, user_id=357e127d423e43fea29162663ecd8666, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:27.954409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:27.966570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 985/1976] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:27 2022] PUT /identity/v3/projects/7940b6ff003d4ba89e7b65425996eede/users/357e127d423e43fea29162663ecd8666/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:27.976698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:27.976994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:27.977279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.003849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] MFA Rules not processed for user `357e127d423e43fea29162663ecd8666`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:28.005116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:28.073199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 992/1977] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:28.084298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:28.086022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:28.147768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0528c8-6496-4acf-9f7e-f4d748e95bd8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:28.151587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:28.152372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.152731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.153079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.161631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:28.163160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db789c-1d99-4ee8-b751-af73660de304 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:28.173080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 986/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:28.194616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:28.257873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faf49b9-ccb2-4b66-bb64-08ad92155aaa None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:28.262449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:28.263313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.263615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.264407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] PATH_INFO: `/v3/users/357e127d423e43fea29162663ecd8666` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.265598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=357e127d423e43fea29162663ecd8666)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:28.267765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:28.289869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:28.317643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '357e127d423e43fea29162663ecd8666'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.326210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '357e127d423e43fea29162663ecd8666'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.327444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '357e127d423e43fea29162663ecd8666'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.335370 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '357e127d423e43fea29162663ecd8666'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.348034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '357e127d423e43fea29162663ecd8666'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.354254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 993/1979] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:28 2022] DELETE /identity/v3/users/357e127d423e43fea29162663ecd8666 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:28.358729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.359022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.359301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.376563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:28.377763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:28.408068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:28.471383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 987/1980] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:28.480708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:28.483028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:28.555928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677fde-9545-4a21-8693-ec1b2407c5ba admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:28.559735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:28.560415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.560731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.561015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.570277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:28.571711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fb1834-b93c-4719-b812-b03d4a4c6b01 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:28.581137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 994/1981] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:28.846037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:28.847460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:28.860343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6327f37-a1d2-43af-ba35-c8683f3b4791 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:28.865167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:28.866062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.866360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.866626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] PATH_INFO: `/v3/projects/7940b6ff003d4ba89e7b65425996eede` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.867818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7940b6ff003d4ba89e7b65425996eede)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:28.869904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:28.916909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7940b6ff003d4ba89e7b65425996eede', 'request_id': 'req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.922911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7940b6ff003d4ba89e7b65425996eede', 'request_id': 'req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.929344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7940b6ff003d4ba89e7b65425996eede', 'request_id': 'req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.942287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] The token cache is being invalidate because project 7940b6ff003d4ba89e7b65425996eede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:28.942576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.943195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a0a06c19-5473-4aaa-a1c0-e439d1beecc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:28.944757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 988/1982] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:28 2022] DELETE /identity/v3/projects/7940b6ff003d4ba89e7b65425996eede => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:28.949868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:28.950346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:28.950575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:28.968149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:28.969393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:29.028726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 995/1983] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:29.037677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.037968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.038250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.056929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:29.058121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:29.067367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 989/1984] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:29.071893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.081550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4210ea-aa3e-4a65-bf8c-9f01c10e7f38 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.086072 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.086723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.087008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.087284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.088012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:29.088355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.089945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.101895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 996/1985] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:29.105931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.115779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ecc376-d035-44aa-9ad8-0e7fb02dbfd5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.120485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.121130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.121417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.121697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.122388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.124327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.150070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 990/1986] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:29 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:29.154081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.164780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead413e6-8294-4d94-bf80-278014761eb8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.197250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:29.230926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.231741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.232030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.232426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.233713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.235626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.260418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 997/1987] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:29 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:29.262131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.273201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17e2c-5dcd-45bd-aa9c-93fe896b9256 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.278414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.279174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.279940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.280256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.281002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.282620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.296644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 991/1988] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:29.300687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.312219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bc6f5-3061-4a13-96e6-10719bb502dc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.317282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.317869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.318159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.318443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.319178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.320830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdd7fd-48e4-44a3-a351-f01e9f3a08b9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.335559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 998/1989] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:29.337001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.347165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d8d18-187b-45b2-b3b3-501e34a2dfaa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.352213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.353099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.353391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.353670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] PATH_INFO: `/v3/projects/b0d2396e024f4311b7b29eed1e91c2f2/users/64eb8a17e0a446b69459c67f59e37bf6/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.368694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0d2396e024f4311b7b29eed1e91c2f2, user_id=64eb8a17e0a446b69459c67f59e37bf6, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.371384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.385779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 992/1990] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:29 2022] PUT /identity/v3/projects/b0d2396e024f4311b7b29eed1e91c2f2/users/64eb8a17e0a446b69459c67f59e37bf6/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:29.396692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.396993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.397281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.424588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] MFA Rules not processed for user `64eb8a17e0a446b69459c67f59e37bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:29.425794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:29.488874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 999/1991] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:29.500431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:29.502122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:29.560030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f1c1-ea64-4f55-b868-8f83ee0ff85a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:29.564013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:29.564681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:29.564972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:29.565255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:29.570610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:29.575832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:29.578891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c2f9a-f7e2-4de3-a669-9cbb82e57c52 service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:29.589341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 993/1992] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:37.006709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:37.008470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:37.076129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50daa8-2b9d-4a72-a40f-8447123822bc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:37.080679 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:37.081552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:37.081842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:37.082121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] PATH_INFO: `/v3/users/d3729ae8e9f7417b85d33dfabedc8c1a` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:37.083307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3729ae8e9f7417b85d33dfabedc8c1a)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:37.085255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:37.141406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:37.148134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3729ae8e9f7417b85d33dfabedc8c1a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.160392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3729ae8e9f7417b85d33dfabedc8c1a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.161144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3729ae8e9f7417b85d33dfabedc8c1a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.170975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3729ae8e9f7417b85d33dfabedc8c1a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.187291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3729ae8e9f7417b85d33dfabedc8c1a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.203027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1000/1993] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:37 2022] DELETE /identity/v3/users/d3729ae8e9f7417b85d33dfabedc8c1a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:37.204564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:37.205061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:37.205147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:37.222936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:37.224191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:37.296714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 994/1994] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:37.311872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:37.313652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:37.371066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26749005-78a5-4fe1-8bf7-4838b5089403 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:37.374866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:37.375517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:37.375807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:37.376092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:37.386625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:37.387957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6958102-9a40-4aa8-ac72-821c830d473c service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:37.398732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1001/1995] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:37.817809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:37.821863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:37.847792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe38f95-45b9-43a0-a78d-b5b28f92a96c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:37.872922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:37.873825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:37.874115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:37.874402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] PATH_INFO: `/v3/projects/85476dbcd1f84580a6fbb1c76f1881b2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:37.876343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=85476dbcd1f84580a6fbb1c76f1881b2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:37.878284 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:37.974640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85476dbcd1f84580a6fbb1c76f1881b2', 'request_id': 'req-1b087b5e-bac2-495e-8900-3cb46a864c6f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.984826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85476dbcd1f84580a6fbb1c76f1881b2', 'request_id': 'req-1b087b5e-bac2-495e-8900-3cb46a864c6f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:37.994696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85476dbcd1f84580a6fbb1c76f1881b2', 'request_id': 'req-1b087b5e-bac2-495e-8900-3cb46a864c6f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:38.009313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] The token cache is being invalidate because project 85476dbcd1f84580a6fbb1c76f1881b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:38.009602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:38.010262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1b087b5e-bac2-495e-8900-3cb46a864c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:38.012430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 995/1996] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:37 2022] DELETE /identity/v3/projects/85476dbcd1f84580a6fbb1c76f1881b2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:38.019046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.019544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.019633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.041547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:38.044230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:38.111628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1002/1997] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:38.120615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.121022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.121307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.148090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:38.149349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:38.162073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 996/1998] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:38.163281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.173456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7a5e4-8e5f-4e9a-9903-34ec42204e16 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.178674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.179444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.180908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.180908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.180908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:38.181446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.183156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.200074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1003/1999] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:38.206451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.216965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447028b-0fb6-4c71-8616-f0d8e6d656ad None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.222311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.222860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.223245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.223755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.224551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.226708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.266265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 997/2000] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:38 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:38.270646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.283449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f4f3-0be4-48f5-911f-fda0b420b92b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.331165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:38.345013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.345688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.345979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.346262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.346966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.348979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.377445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1004/2001] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:38 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:38.381496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.393038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2602f04-3507-4515-8a79-3be735df0d60 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.398151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.398790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.399068 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.399347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.400071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.407378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.421750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 998/2002] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:08:38.427880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.451437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6460a9-cefb-495e-b5ee-3f117722623f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.457628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.458416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.459055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.459506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.460964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.463326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2236c7-497a-4a6f-87d3-feea5661a2f8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.485541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1005/2003] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:38.486132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.497226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636feab9-80ca-4dfc-a79d-bc631668a149 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.503053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.504061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.504473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.504849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] PATH_INFO: `/v3/projects/1583bd5823544e54b32e1eaf308cd261/users/0dbf681122194384812c2eb96d1adfbc/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.531003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1583bd5823544e54b32e1eaf308cd261, user_id=0dbf681122194384812c2eb96d1adfbc, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.534325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.552145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 999/2004] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:38 2022] PUT /identity/v3/projects/1583bd5823544e54b32e1eaf308cd261/users/0dbf681122194384812c2eb96d1adfbc/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:38.556933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.557226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.557507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.592478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:38.593739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:38.677570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1006/2005] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:38.704692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:38.707505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:38.725796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:38.779570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb01a39-19ac-4936-8892-49e31d3a34a9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:38.783291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:38.783882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:38.784196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:38.784477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:38.795816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:38.797290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf9af0-57e6-4af6-93b9-a7bdfa76011e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:38.807326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1000/2006] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:08:39.889685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:39.892076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:39.906149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc89f8b-230b-4a26-b9e3-e94fc12f3233 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:39.911670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:39.912600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:39.912885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:39.913167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] PATH_INFO: `/v3/users/b974bab2e7934b57ab79128210c439a7` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:39.914350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b974bab2e7934b57ab79128210c439a7)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:39.916491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:39.979345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b974bab2e7934b57ab79128210c439a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:39.984061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:39.984573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:39.984681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:39.988766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b974bab2e7934b57ab79128210c439a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:39.989636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b974bab2e7934b57ab79128210c439a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:39.998560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b974bab2e7934b57ab79128210c439a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.012606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b974bab2e7934b57ab79128210c439a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.012847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] MFA Rules not processed for user `0dbf681122194384812c2eb96d1adfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:40.014723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:40.024177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1007/2007] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:39 2022] DELETE /identity/v3/users/b974bab2e7934b57ab79128210c439a7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:40.148907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1001/2008] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:40.160048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.161789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:40.221831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:40.252395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.253990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:40.275698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1767d-5a90-4ed5-9822-d448508a211d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.275698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.277879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ac779-488f-4237-b32e-87e5b832f685 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.278636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.279620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.280024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.362127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.363565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ceea6-b1d5-4296-8446-c19a741257be service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.375314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.376412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.376648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.376997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] PATH_INFO: `/v3/projects/96ee20ba4bcd469097e1a8815eee16e6` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.378399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ee20ba4bcd469097e1a8815eee16e6)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.379474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1008/2009] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:08:40.380740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.445507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ee20ba4bcd469097e1a8815eee16e6', 'request_id': 'req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.453191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ee20ba4bcd469097e1a8815eee16e6', 'request_id': 'req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.454679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:40.461621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ee20ba4bcd469097e1a8815eee16e6', 'request_id': 'req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.474378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] The token cache is being invalidate because project 96ee20ba4bcd469097e1a8815eee16e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:40.474665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.475304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-402cad84-e150-40b2-bb8e-9a4ccf7e63c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:40.477116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1002/2010] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:40 2022] DELETE /identity/v3/projects/96ee20ba4bcd469097e1a8815eee16e6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:40.482172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.482571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.482857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.508475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:40.509854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:40.602035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1009/2011] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:40.605266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.605661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.606377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.630286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:40.631789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:40.647758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1003/2012] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:40.653385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.668136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4eb97-e3e1-40a2-93de-c80f782b930d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.681616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.682291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.682592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.682972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.684002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:40.684268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.686501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.711948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1010/2013] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:08:40.715911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.727583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5c4c9-ff2e-4d12-997b-4704aa09a4cb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.733820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.734570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.734981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.735305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.736583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.739567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.815765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1004/2014] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:40 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:40.817873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.832285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f931-64c2-47bc-bae1-b897c5cf3c7a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.906100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.906778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.907087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.907413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.908082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.910117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.934132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1011/2015] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:40 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:40.938957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.949423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff83c5-214a-4aea-99e2-232dfd3f554e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.954066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.954779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.955073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.955353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.956080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.957715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.968517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1005/2016] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:40.972298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:40.982622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3478b-c413-4ac8-9445-9dfebcfc8084 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:40.987187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:40.987835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:40.988145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:40.988426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:40.989140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:40.990725 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5322a-0257-4882-89e0-de7e7f504dd3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:40.995848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:41.002355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1012/2017] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:41.006258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.016324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51766ad9-bec0-42e7-b4c3-b5a328cf6502 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.021182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.022052 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.022345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.022626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] PATH_INFO: `/v3/projects/4e930e7ec0aa49d3b740cae83ec4feb6/users/b03bc6cad0b144d1806ebc518fd12235/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.037104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e930e7ec0aa49d3b740cae83ec4feb6, user_id=b03bc6cad0b144d1806ebc518fd12235, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.039762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.054310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1006/2018] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:41 2022] PUT /identity/v3/projects/4e930e7ec0aa49d3b740cae83ec4feb6/users/b03bc6cad0b144d1806ebc518fd12235/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:41.065922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.066241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.066526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.093707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] MFA Rules not processed for user `b03bc6cad0b144d1806ebc518fd12235`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:41.094912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:41.159968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1013/2019] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:41.164553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.228808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e546892-d3cc-47a9-8550-8919683029dd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.234263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.234941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.235224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.235502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.236253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.238164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.263167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1007/2020] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:41 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:41.267484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.278315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303329-d259-4042-be02-cab3dfd75f3a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.334774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.335588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.336005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.336322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.337027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.343441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.361698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1014/2021] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:41 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:41.365790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.376076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08841e00-d5d8-4e17-ad95-d03c63cfc05f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.380610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.381396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.381682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.381956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.382684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.384323 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.395277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1008/2022] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:41.399501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.410220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c6f46-d050-40b5-b822-917ed9c5ea3c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.414821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.415720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.415988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.416313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] PATH_INFO: `/v3/projects/c2133329f44449209758fe046855d76f/users/c2860f03c01746e796d684a6c8f0243c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.431951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2133329f44449209758fe046855d76f, user_id=c2860f03c01746e796d684a6c8f0243c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.434801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.449040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1015/2023] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:41 2022] PUT /identity/v3/projects/c2133329f44449209758fe046855d76f/users/c2860f03c01746e796d684a6c8f0243c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:41.453528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.464363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b4a9c1-b7d5-40e9-958a-a7f89228a361 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.508743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:41.530357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.531313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.531837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.532170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.533286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.535800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334d769-91c5-4e84-a483-70b9182dcb09 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.546880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1009/2024] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:41.551200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.561576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4a13-af51-4440-bee3-69546f0da2de admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.566184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.567070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.567371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.567635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] PATH_INFO: `/v3/projects/c2133329f44449209758fe046855d76f/users/c2860f03c01746e796d684a6c8f0243c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.569632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2133329f44449209758fe046855d76f, user_id=c2860f03c01746e796d684a6c8f0243c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.572284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.584827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1016/2025] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:41 2022] PUT /identity/v3/projects/c2133329f44449209758fe046855d76f/users/c2860f03c01746e796d684a6c8f0243c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:08:41.594752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.595047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.595326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.640388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] MFA Rules not processed for user `c2860f03c01746e796d684a6c8f0243c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:41.641963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:41.713511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1010/2026] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:41.725095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.726801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:41.773696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:41.786563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db641b6b-b8c5-4e9f-8fd4-1d2f84ac343f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.790192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.790829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.791111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.791386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.802300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.803745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf29cf-d60f-46c7-8efc-08db72590c1c service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.813799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1017/2027] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:41.886331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:41.898006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bf8c2-3b18-4bc4-8f92-569de35f473c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:41.901865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:41.902958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.903320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.903767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.910468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:41.911005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:41.911121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:41.929111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] MFA Rules not processed for user `0dbf681122194384812c2eb96d1adfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:41.930351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:41.966873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:41.968400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf332fc2-36bd-4b23-b035-ae6e9b61aa75 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:41.982737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1011/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:41.988856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1018/2029] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:08:51.537152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b173ef-bd55-4eed-b35b-6ebd7d2505db None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:51.537735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b173ef-bd55-4eed-b35b-6ebd7d2505db None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:51.537735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b173ef-bd55-4eed-b35b-6ebd7d2505db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:51.558191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-04b173ef-bd55-4eed-b35b-6ebd7d2505db None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:51.559575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-04b173ef-bd55-4eed-b35b-6ebd7d2505db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:51.621342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1012/2030] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:08:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:51.668842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:51.671596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:51.758331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31501f-f8c0-4fd8-8e1b-e2b715ee08ac None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:51.763521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:51.764308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:51.764481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:51.764763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:51.774096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:51.775691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:51.784748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1019/2031] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:08:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:08:52.363592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1260d6d-4bf9-42b0-a103-ce1da78bd686 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:52.363888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1260d6d-4bf9-42b0-a103-ce1da78bd686 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:52.364252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1260d6d-4bf9-42b0-a103-ce1da78bd686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:52.384017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a1260d6d-4bf9-42b0-a103-ce1da78bd686 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:52.385320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a1260d6d-4bf9-42b0-a103-ce1da78bd686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:52.393644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1013/2032] 10.222.0.23 () {58 vars in 967 bytes} [Mon Jun 20 07:08:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:52.406277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:52.417191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556099d-6cb6-4f49-8f1f-3d02dfa25072 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:52.423090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:52.423760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:52.424053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:52.424371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:52.432974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:52.434765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:52.443451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1020/2033] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:08:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:08:52.653458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ecf9a-5713-4910-a637-720f7217b25e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:52.653995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ecf9a-5713-4910-a637-720f7217b25e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:52.654082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ecf9a-5713-4910-a637-720f7217b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:52.660905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:52.672718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-647ecf9a-5713-4910-a637-720f7217b25e None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:52.673899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-647ecf9a-5713-4910-a637-720f7217b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:52.681018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1014/2034] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:08:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:52.693349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:52.703576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c5ad-3524-4f96-bd5c-233e877bb226 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:52.708605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8b42-5527-45da-b443-943bda451112 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:52.709244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8b42-5527-45da-b443-943bda451112 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:52.709866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8b42-5527-45da-b443-943bda451112 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:52.709866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873e8b42-5527-45da-b443-943bda451112 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:52.718125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e8b42-5527-45da-b443-943bda451112 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:52.719640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873e8b42-5527-45da-b443-943bda451112 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:52.727565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1021/2035] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:08:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:08:53.069481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:53.069980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:53.070194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:53.087847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:53.089113 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:53.096781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1015/2036] 10.222.0.23 () {58 vars in 967 bytes} [Mon Jun 20 07:08:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:08:53.109515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8b42-5527-45da-b443-943bda451112 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:53.119913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873e8b42-5527-45da-b443-943bda451112 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:53.125412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:53.126373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:53.126805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:53.127227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:53.135975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:53.137694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f32cfe-2602-4b5d-a230-9cf17b9f972e service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:53.146061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1022/2037] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:08:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:08:58.577138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:58.581546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:58.648978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35935479-6d5d-4c34-baf2-4630b91f505d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:58.653558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:58.654430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:58.654716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:58.654993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] PATH_INFO: `/v3/users/fc02ed62c23443ab9f968a71c2c674bf` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:58.656230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc02ed62c23443ab9f968a71c2c674bf)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:58.658178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:58.712928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc02ed62c23443ab9f968a71c2c674bf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:58.713926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc02ed62c23443ab9f968a71c2c674bf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:58.714686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc02ed62c23443ab9f968a71c2c674bf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:58.721849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc02ed62c23443ab9f968a71c2c674bf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:58.734890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc02ed62c23443ab9f968a71c2c674bf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:58.747092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1016/2038] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:08:58 2022] DELETE /identity/v3/users/fc02ed62c23443ab9f968a71c2c674bf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:58.752331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:58.752821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:58.752939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:58.770520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:58.771724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:58.809133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:58.845671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1023/2039] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:58.853793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:58.855316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:58.925030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38e6bc-637d-487b-9c44-c214283f9ae2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:58.930199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:58.931004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:58.931291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:58.931562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:58.942506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:58.943930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7320a-e463-49f4-ac1d-1bae7f6cd447 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:58.955330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1017/2040] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:08:59.152569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.154855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:59.177222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402790c-2c35-42f9-a381-ffa4862f8f95 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.190350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.192243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.192550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.192870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] PATH_INFO: `/v3/projects/20a5d413451c4fe79fb8a521ddf29326` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.194399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a5d413451c4fe79fb8a521ddf29326)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.196505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.329816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a5d413451c4fe79fb8a521ddf29326', 'request_id': 'req-d7def15e-3b24-457a-99c1-c76a824aad9c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:59.347494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a5d413451c4fe79fb8a521ddf29326', 'request_id': 'req-d7def15e-3b24-457a-99c1-c76a824aad9c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:59.358863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a5d413451c4fe79fb8a521ddf29326', 'request_id': 'req-d7def15e-3b24-457a-99c1-c76a824aad9c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:59.384694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] The token cache is being invalidate because project 20a5d413451c4fe79fb8a521ddf29326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:08:59.385773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:59.387719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d7def15e-3b24-457a-99c1-c76a824aad9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:08:59.393247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1024/2041] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:08:59 2022] DELETE /identity/v3/projects/20a5d413451c4fe79fb8a521ddf29326 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:08:59.398101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.399985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.400296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.421763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:59.423158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:59.435242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:08:59.515663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1018/2042] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:59.525531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.525900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.526270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.562061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:59.562426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:59.591596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1025/2043] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:59.595467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.611869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f0777-ca71-4ab5-9e34-2d855e48b773 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.624150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.624846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.625136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.625416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.626163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:08:59.626482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.628177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.662370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1019/2044] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:08:59.666772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.679432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2745d2-172b-4592-b1df-702a505c4d25 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.699825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.700779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.701794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.702242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.703115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.705582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.771091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1026/2045] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:08:59 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:59.772657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.789330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831faef7-f866-4933-82ea-b976cfbd5c59 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.874193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.874904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.875196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.875479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.876244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.878288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a53b96-daf7-48bf-83e8-3e4daa232aa0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.902945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1020/2046] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:08:59 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:08:59.905245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.910816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b598f7-462f-4d06-9ec7-2ff0fb65998e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.911556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b598f7-462f-4d06-9ec7-2ff0fb65998e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.911556 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b598f7-462f-4d06-9ec7-2ff0fb65998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.920508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259cbc76-8e5d-475a-a1ef-eef6fd1bb805 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.925300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.926386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.926671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.926951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.927687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.929763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.932475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a2b598f7-462f-4d06-9ec7-2ff0fb65998e None None] MFA Rules not processed for user `f66183ca9d884f43befaace02099c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:08:59.933705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a2b598f7-462f-4d06-9ec7-2ff0fb65998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:08:59.946713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1027/2047] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:08:59.950973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.961990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ffdfe-f368-4721-9107-542e9f5783d9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:08:59.967499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:08:59.968181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:08:59.968465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:08:59.968744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:08:59.969496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:08:59.971138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:08:59.985198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1028/2048] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:08:59.986148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:08:59.995550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1021/2049] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:08:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:08:59.995975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36904c2-a83a-44f8-8504-7e0e269231d6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:00.005140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:00.006082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:00.006356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:00.006629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] PATH_INFO: `/v3/projects/f5a98ee7cefc4c759c846f6cacb646a8/users/1404717904a14000a11f06c6c8291105/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:00.021945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a98ee7cefc4c759c846f6cacb646a8, user_id=1404717904a14000a11f06c6c8291105, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:00.025268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:00.042125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1029/2050] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:08:59 2022] PUT /identity/v3/projects/f5a98ee7cefc4c759c846f6cacb646a8/users/1404717904a14000a11f06c6c8291105/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:09:00.048349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5aa3-1efa-4de0-9149-a0e765694fd1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:00.048903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5aa3-1efa-4de0-9149-a0e765694fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:00.049543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c5aa3-1efa-4de0-9149-a0e765694fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:00.069023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-601c5aa3-1efa-4de0-9149-a0e765694fd1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:00.070235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-601c5aa3-1efa-4de0-9149-a0e765694fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:00.112534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:00.150413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1022/2051] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:00.159687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:00.161529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:00.215637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:00.220408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db06086-f2ca-4077-8a59-2ffec53d8f97 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:00.225536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:00.226206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:00.227254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:00.227531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:00.237983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:00.239436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:00.251140 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1030/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:05.657054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:05.658058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:05.658532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:05.705281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] MFA Rules not processed for user `1404717904a14000a11f06c6c8291105`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:05.707708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:05.853831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1023/2053] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:05.859399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:05.873237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acaf7e9-bf07-4cbe-82a8-8a1255329dee service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:05.897342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:05.898259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:05.901877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:05.904128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:05.905649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:05.910618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:05.977015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1031/2054] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:05 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:05.979262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:06.004425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c5c1-412f-41ab-98d3-66194dd045ff None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:06.174766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:06.175839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:06.176661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:06.177372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:06.178335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:06.181267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:06.213634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1024/2055] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:05 2022] POST /identity/v3/users => generated 296 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:06.217944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:06.243564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bf0b6-b73a-4537-b8e4-f67aa82a968e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:06.253925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:06.254761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:06.254930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:06.257563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:06.260722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:06.263851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:06.284760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1032/2056] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:06.288778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:06.306180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a93aeb-be93-4569-a8fc-16e046e37245 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:06.313278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:06.313957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:06.314245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:06.314529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] PATH_INFO: `/v3/projects/00f0938b938948fda228f78f1d06d071/users/66c18d8f9715494eb53e38ffd5c9531d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:06.333559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f0938b938948fda228f78f1d06d071, user_id=66c18d8f9715494eb53e38ffd5c9531d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:06.336995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:06.366024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1025/2057] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:06 2022] PUT /identity/v3/projects/00f0938b938948fda228f78f1d06d071/users/66c18d8f9715494eb53e38ffd5c9531d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:06.370758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:06.387008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22c461-5314-4ab9-bd33-b808c743d963 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:06.483714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:06.484514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:06.484849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:06.485162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:06.485925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:06.487684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:06.500254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1033/2058] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:06.504281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:06.524406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b5577-4e9b-4b29-887c-dcf94ffd279f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:06.534191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:06.535249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:06.535724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:06.536267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] PATH_INFO: `/v3/projects/00f0938b938948fda228f78f1d06d071/users/66c18d8f9715494eb53e38ffd5c9531d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:06.540525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f0938b938948fda228f78f1d06d071, user_id=66c18d8f9715494eb53e38ffd5c9531d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:06.543663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760f56d-59a3-498f-b431-6291856c795b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:06.557175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1026/2059] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:06 2022] PUT /identity/v3/projects/00f0938b938948fda228f78f1d06d071/users/66c18d8f9715494eb53e38ffd5c9531d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:09:11.152258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:11.155735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:11.177802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac30d02-589a-456d-b8af-0d81cde5b9be admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:11.297417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:11.299314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:11.299633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:11.299910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] PATH_INFO: `/v3/users/64eb8a17e0a446b69459c67f59e37bf6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:11.301762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64eb8a17e0a446b69459c67f59e37bf6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:11.304134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:11.369161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64eb8a17e0a446b69459c67f59e37bf6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:11.378863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64eb8a17e0a446b69459c67f59e37bf6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:11.379767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64eb8a17e0a446b69459c67f59e37bf6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:11.382247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:11.401020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64eb8a17e0a446b69459c67f59e37bf6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:11.424360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64eb8a17e0a446b69459c67f59e37bf6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:11.432201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1034/2060] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:11 2022] DELETE /identity/v3/users/64eb8a17e0a446b69459c67f59e37bf6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:11.436701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:11.437286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:11.437286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:11.472293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:11.473656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:11.524714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:11.603078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1027/2061] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:11.615965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:11.619521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:11.799311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f1b28e-8a82-401c-bece-ccd9ec0c7952 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:11.810076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:11.814201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:11.814702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:11.815157 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:11.839025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:11.842605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3f5c9-bcc1-40b3-92e2-60e7b850c5e5 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:11.856935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1035/2062] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:09:12.338737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:12.341097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:12.362095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0727cbc-7cd9-43f7-98c6-3aad6ed734bb None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:12.379571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:12.380546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.380763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.381055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] PATH_INFO: `/v3/projects/b0d2396e024f4311b7b29eed1e91c2f2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.382499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0d2396e024f4311b7b29eed1e91c2f2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:12.384455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:12.460009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0d2396e024f4311b7b29eed1e91c2f2', 'request_id': 'req-803bb29c-a5ba-4933-a0cf-1d686c670180'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:12.468136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0d2396e024f4311b7b29eed1e91c2f2', 'request_id': 'req-803bb29c-a5ba-4933-a0cf-1d686c670180'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:12.477076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0d2396e024f4311b7b29eed1e91c2f2', 'request_id': 'req-803bb29c-a5ba-4933-a0cf-1d686c670180'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:12.499399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] The token cache is being invalidate because project b0d2396e024f4311b7b29eed1e91c2f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:12.499809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:12.500776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-803bb29c-a5ba-4933-a0cf-1d686c670180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:12.502731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1028/2063] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:12 2022] DELETE /identity/v3/projects/b0d2396e024f4311b7b29eed1e91c2f2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:12.509558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.510446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.510902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.547201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:12.549356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:12.669816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1036/2064] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:12.677614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.677893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.678360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.706278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:12.708469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:12.721336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1029/2065] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:12.725553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:12.743814 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b763529-7e6f-4acd-b027-6131b743020f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:12.753567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:12.754555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.755052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.755521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.756760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:09:12.757251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:12.759224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:12.774824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1037/2066] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:09:12.779394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:12.790662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ae29d-a832-4483-b4c8-c3569feb14de None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:12.813758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:12.814725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.815159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.815592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.816584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:12.819093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:12.862106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1030/2067] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:12 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:12.866667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:12.882288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999c7ba0-1a55-4cdf-b3c0-69d6b343f0f8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:12.954579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:12.955278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:12.955568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:12.955849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:12.956605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:12.959049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:12.970513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:12.994091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1038/2068] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:12 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:12.998997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.013024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e07ff-b916-4991-84e4-cbdf7bcbfbab admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:13.018978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:13.020803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.021419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.021974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.022849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:13.024727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:13.039912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1031/2069] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:13.043718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.055614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267664c-de85-424f-ad60-043d07b6e3c4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:13.061479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:13.062554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.062848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.063139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.063901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:13.066286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fe6c5-ef50-48c7-a80d-150bc26a70e6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:13.081113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1039/2070] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:13.083668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.096454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e763a7-efc2-4fd2-85be-c07ab30a4ee5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:13.101631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:13.102788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.103074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.103354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] PATH_INFO: `/v3/projects/b28934c608544b5da9b2fe73a929df20/users/3b6836254b6948d7931e6e2e70b3dea6/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.122521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b28934c608544b5da9b2fe73a929df20, user_id=3b6836254b6948d7931e6e2e70b3dea6, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:13.125470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:13.149782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1032/2071] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:13 2022] PUT /identity/v3/projects/b28934c608544b5da9b2fe73a929df20/users/3b6836254b6948d7931e6e2e70b3dea6/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:13.162138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.162811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.163113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.201804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] MFA Rules not processed for user `3b6836254b6948d7931e6e2e70b3dea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:13.203091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:13.299503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1040/2072] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:13.307586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.310339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:13.408193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:13.429334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dc313-2bab-4500-85f1-27c8abbbcd76 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:13.434930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:13.435643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.435940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.436263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.449363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:13.450826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df18cd-0fca-4f72-a850-95ac8ee95c07 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:13.468241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1033/2073] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:13.489052 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.571512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116ff8e-ce48-43d0-af70-b474e36e5fec None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:13.577962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:13.578947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.579305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.579653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] PATH_INFO: `/v3/users/3b6836254b6948d7931e6e2e70b3dea6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.581026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b6836254b6948d7931e6e2e70b3dea6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:13.583441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:13.638401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b6836254b6948d7931e6e2e70b3dea6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:13.659740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b6836254b6948d7931e6e2e70b3dea6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:13.660643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b6836254b6948d7931e6e2e70b3dea6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:13.677189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b6836254b6948d7931e6e2e70b3dea6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:13.694001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b6836254b6948d7931e6e2e70b3dea6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:13.701417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:13.704450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1041/2074] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:13 2022] DELETE /identity/v3/users/3b6836254b6948d7931e6e2e70b3dea6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:13.708583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1a88-991d-48f1-8f45-95519c04917f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:13.708874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1a88-991d-48f1-8f45-95519c04917f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:13.709152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1a88-991d-48f1-8f45-95519c04917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:13.733550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bdde1a88-991d-48f1-8f45-95519c04917f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:13.735097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bdde1a88-991d-48f1-8f45-95519c04917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:13.857812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1034/2075] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:09:13.874639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:13.876432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.008581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9f5c5-4b57-4254-ada8-5790cd478c6b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:14.019552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:14.021023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.021880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.022170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.035673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:14.037960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:14.060793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.061116 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.061402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.065933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1042/2076] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:14.108312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] MFA Rules not processed for user `66c18d8f9715494eb53e38ffd5c9531d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:14.109563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.220373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1035/2077] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:14.232687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:14.235000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.384885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e4526-394d-434c-857d-5719c45c0604 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:14.392504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:14.392504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.392712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.392968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.394967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.497525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:14.501536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b89538-d264-41f1-b8b2-97ae59d104bd service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:14.518417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1043/2078] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:14.570732 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:14.576011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.618055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b042e9-a609-4c88-b183-a5f54e026259 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:14.635613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:14.635613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.635613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.635613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] PATH_INFO: `/v3/projects/b28934c608544b5da9b2fe73a929df20` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.636046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b28934c608544b5da9b2fe73a929df20)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:14.637543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:14.728859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b28934c608544b5da9b2fe73a929df20', 'request_id': 'req-98953b97-65f7-459f-bf60-6f7c8f0674f5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:14.746347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b28934c608544b5da9b2fe73a929df20', 'request_id': 'req-98953b97-65f7-459f-bf60-6f7c8f0674f5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:14.750636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:14.762007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b28934c608544b5da9b2fe73a929df20', 'request_id': 'req-98953b97-65f7-459f-bf60-6f7c8f0674f5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:14.789731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] The token cache is being invalidate because project b28934c608544b5da9b2fe73a929df20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:14.790025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:14.793167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-98953b97-65f7-459f-bf60-6f7c8f0674f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:14.795332 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1036/2079] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:14 2022] DELETE /identity/v3/projects/b28934c608544b5da9b2fe73a929df20 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:14.800518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee441c-1641-4064-a030-52543f47a845 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.801276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee441c-1641-4064-a030-52543f47a845 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.801276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee441c-1641-4064-a030-52543f47a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.826910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-19ee441c-1641-4064-a030-52543f47a845 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:14.828627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-19ee441c-1641-4064-a030-52543f47a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.875583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:14.930639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1044/2080] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:09:14.932474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:14.932890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:14.933258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:14.960051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:14.961289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:14.983171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1037/2081] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:14.987495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee441c-1641-4064-a030-52543f47a845 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:14.997948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee441c-1641-4064-a030-52543f47a845 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.007987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.008737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.009025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.009304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.010040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:09:15.010922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.012298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.029646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1045/2082] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:09:15.031084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.040438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb24f59-968f-4117-86e3-23d01d6c0a04 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.045630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.046285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.046574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.046846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.047548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.049703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.075645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1038/2083] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:15 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:15.079808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.091722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d6705d-3bab-4273-b193-72f683307da1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.179929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.180818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.181106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.181396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.182104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.184039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.213818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1046/2084] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:15 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:15.216251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.228735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4ee50-a3e8-49c5-af62-c9baad58efc7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.234023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.234790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.235155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.235442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.236241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.237917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.249327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1039/2085] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:15.253390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.264257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8a1af-4ff6-4d5d-9d6d-4d2ee0c0b111 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.269263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.269931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.270222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.270505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.271273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.273174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebef93-f336-4b8f-94fc-54bb25b50032 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.287904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1047/2086] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:15.292632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.306769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864f2c1-aa79-46a3-b599-eff2c700264b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.317851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.318918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.319246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.319482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] PATH_INFO: `/v3/projects/87b854afdcf44edb931fff78357ef8fe/users/b7ae7fefb37744c3b402115ac4cc16e8/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.338169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b854afdcf44edb931fff78357ef8fe, user_id=b7ae7fefb37744c3b402115ac4cc16e8, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.341519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.364242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1040/2087] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:15 2022] PUT /identity/v3/projects/87b854afdcf44edb931fff78357ef8fe/users/b7ae7fefb37744c3b402115ac4cc16e8/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:15.371835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e195a-c389-43ed-af58-17300707285d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.372160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e195a-c389-43ed-af58-17300707285d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.372450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e195a-c389-43ed-af58-17300707285d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.406595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-386e195a-c389-43ed-af58-17300707285d None None] MFA Rules not processed for user `b7ae7fefb37744c3b402115ac4cc16e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:15.408131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-386e195a-c389-43ed-af58-17300707285d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:15.492216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1048/2088] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:15.501396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:15.503396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:15.602976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4fe21-2f6f-4209-9dc0-71e82e10def1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:15.611834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:15.612544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:15.613490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:15.613887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:15.626376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:15.637211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f56e5-d7e8-4655-9200-c1ea49ae42b3 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:15.650383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1041/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:25.487494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:25.644618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:31.786948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e195a-c389-43ed-af58-17300707285d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:32.004731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e195a-c389-43ed-af58-17300707285d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:32.010774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:32.012422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:32.012954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:32.013701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] PATH_INFO: `/v3/users/b7ae7fefb37744c3b402115ac4cc16e8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:32.016078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ae7fefb37744c3b402115ac4cc16e8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:32.018790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:32.115899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ae7fefb37744c3b402115ac4cc16e8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:32.125274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ae7fefb37744c3b402115ac4cc16e8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:32.126173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ae7fefb37744c3b402115ac4cc16e8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:32.141166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ae7fefb37744c3b402115ac4cc16e8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:32.161640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ae7fefb37744c3b402115ac4cc16e8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:32.171419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1049/2090] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:31 2022] DELETE /identity/v3/users/b7ae7fefb37744c3b402115ac4cc16e8 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:32.176812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:32.177378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:32.177609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:32.252055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:32.253356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:32.344583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1042/2091] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:32.357764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:32.360501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:32.423023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:32.486479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26109f-06d8-4403-a80c-45e74fb1a861 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:32.490266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:32.490946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:32.491264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:32.491543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:32.502024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:32.503490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce323d-a57e-4b96-bfee-bbd165dd97c9 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:32.516092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1050/2092] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:32.963378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:32.963378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:32.987708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91d10e-623b-46a7-8a21-a1cc3b8ca026 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:33.007440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:33.008357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:33.008663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:33.008926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] PATH_INFO: `/v3/projects/87b854afdcf44edb931fff78357ef8fe` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:33.010247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b854afdcf44edb931fff78357ef8fe)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:33.012349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:33.082525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b854afdcf44edb931fff78357ef8fe', 'request_id': 'req-3d9a1488-163f-43a0-8b93-39f22a87b899'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.088872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b854afdcf44edb931fff78357ef8fe', 'request_id': 'req-3d9a1488-163f-43a0-8b93-39f22a87b899'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.096283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b854afdcf44edb931fff78357ef8fe', 'request_id': 'req-3d9a1488-163f-43a0-8b93-39f22a87b899'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.114163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] The token cache is being invalidate because project 87b854afdcf44edb931fff78357ef8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:33.115171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.115912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.118113 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1043/2093] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:32 2022] DELETE /identity/v3/projects/87b854afdcf44edb931fff78357ef8fe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:09:33.328555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476a69-df81-4223-8593-409e71eac4bc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:33.329132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476a69-df81-4223-8593-409e71eac4bc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:33.329228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476a69-df81-4223-8593-409e71eac4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:33.354501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7d476a69-df81-4223-8593-409e71eac4bc None None] MFA Rules not processed for user `af0dee4e1dfb4f879dfe41622a9e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:33.355825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7d476a69-df81-4223-8593-409e71eac4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:33.419290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1051/2094] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:33.616370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:33.619900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:33.697241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:33.703288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a1488-163f-43a0-8b93-39f22a87b899 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:33.715347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:33.716147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:33.716447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:33.716729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] PATH_INFO: `/v3/users/b03bc6cad0b144d1806ebc518fd12235` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:33.718475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03bc6cad0b144d1806ebc518fd12235)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:33.720459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:33.768851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03bc6cad0b144d1806ebc518fd12235'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.769506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03bc6cad0b144d1806ebc518fd12235'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.770115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03bc6cad0b144d1806ebc518fd12235'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.777473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03bc6cad0b144d1806ebc518fd12235'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.790530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03bc6cad0b144d1806ebc518fd12235'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:33.803606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1044/2095] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:33 2022] DELETE /identity/v3/users/b03bc6cad0b144d1806ebc518fd12235 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:33.808160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:33.808529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:33.808873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:33.826312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:33.827535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:33.909656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1052/2096] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:09:33.922521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:33.926246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:34.001163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16df36f-4121-4faf-87ea-3f75e4dfe69c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:34.012880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:34.013602 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:34.014581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:34.014869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:34.026778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:34.029232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:34.050823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1045/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:34.428967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:34.431321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:34.444783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab0da34-74b2-4563-8ec0-13f6afe59075 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:34.450155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:34.451957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:34.452423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:34.452893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] PATH_INFO: `/v3/projects/4e930e7ec0aa49d3b740cae83ec4feb6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:34.454272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e930e7ec0aa49d3b740cae83ec4feb6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:34.456851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:34.525268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:34.528962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e930e7ec0aa49d3b740cae83ec4feb6', 'request_id': 'req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.538082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e930e7ec0aa49d3b740cae83ec4feb6', 'request_id': 'req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.545705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e930e7ec0aa49d3b740cae83ec4feb6', 'request_id': 'req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.558657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] The token cache is being invalidate because project 4e930e7ec0aa49d3b740cae83ec4feb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:34.559016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.559610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.562199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1053/2098] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:34 2022] DELETE /identity/v3/projects/4e930e7ec0aa49d3b740cae83ec4feb6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:34.571375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:34.593017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d4d49-3125-46df-a3a3-e4719922e7f4 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:34.722689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:34.723800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:34.724214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:34.724533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] PATH_INFO: `/v3/users/c2860f03c01746e796d684a6c8f0243c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:34.726189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2860f03c01746e796d684a6c8f0243c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:34.728135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:34.773475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:34.811701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2860f03c01746e796d684a6c8f0243c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.812548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2860f03c01746e796d684a6c8f0243c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.813294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2860f03c01746e796d684a6c8f0243c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.823940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2860f03c01746e796d684a6c8f0243c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.837811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5c9e6e15-e58f-47ee-9e85-e39022afb0b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2860f03c01746e796d684a6c8f0243c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:34.862684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1046/2099] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:34 2022] DELETE /identity/v3/users/c2860f03c01746e796d684a6c8f0243c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:35.156405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.157993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:35.172954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc55c5-bfaf-4b23-83cb-51cc6184ec3e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.236691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.238311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.238604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.238884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] PATH_INFO: `/v3/projects/c2133329f44449209758fe046855d76f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.240339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2133329f44449209758fe046855d76f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.242435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.294715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2133329f44449209758fe046855d76f', 'request_id': 'req-65eea92f-afef-4cdc-bbda-49a0b11622fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:35.305657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2133329f44449209758fe046855d76f', 'request_id': 'req-65eea92f-afef-4cdc-bbda-49a0b11622fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:35.311143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2133329f44449209758fe046855d76f', 'request_id': 'req-65eea92f-afef-4cdc-bbda-49a0b11622fa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:35.322775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] The token cache is being invalidate because project c2133329f44449209758fe046855d76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:35.323204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:35.324045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-65eea92f-afef-4cdc-bbda-49a0b11622fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:35.328336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1054/2100] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:35 2022] DELETE /identity/v3/projects/c2133329f44449209758fe046855d76f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:35.330474 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.330772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.331070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.374239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:35.375453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:35.461053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1047/2101] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:35.470286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.470286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.470764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.495781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:35.497405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:35.507077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1055/2102] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:35.510972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.523296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae1a79-e9df-4051-b5f3-a110217e256c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.532834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.533828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.534384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.534867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.536260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:09:35.537358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.541783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.571137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1048/2103] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:09:35.575349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.588693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d40ad66-bfdc-4524-9020-c053cccfb822 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.595435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.596213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.596503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.596783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.597483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.599766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.636928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1056/2104] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:35.638018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.652250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ce384-a1cb-4479-918d-d4fd1b00db22 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.769330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.769999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.770287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.770567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.771272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.773426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.797542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:35.808404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1049/2105] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:09:35.810308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.831701 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c5f16-79f5-428a-bce2-ba2b35ad9032 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.836538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.837204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.837543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.837960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.838554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.840208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.851459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1057/2106] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:35.855253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.872618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197943e-7a60-491c-8c86-cb26f5791628 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.884851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.885817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.886170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.886617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.887423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.890370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891e3b5-a311-492c-b0b5-ae6fcfb5c930 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.904763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1050/2107] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:35.909168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:35.923438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07775a-b736-454f-8df6-2d25a2bc2bbf admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:35.928447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:35.929590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.929966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.930293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] PATH_INFO: `/v3/projects/8ab7d15c95b74d12b5e20d9bfcc9db67/users/0a5eeece2f084bc5a5b91e03107eeee9/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.956501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab7d15c95b74d12b5e20d9bfcc9db67, user_id=0a5eeece2f084bc5a5b91e03107eeee9, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:35.959870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ab8d9-ccbc-4b16-bd92-7b5b18e0a487 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:35.967238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.967695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.968227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:35.973739 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:35.983086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1058/2108] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:35 2022] PUT /identity/v3/projects/8ab7d15c95b74d12b5e20d9bfcc9db67/users/0a5eeece2f084bc5a5b91e03107eeee9/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:09:35.990461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:35.992354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:35.993720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:35.994133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:35.994419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.024935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] MFA Rules not processed for user `0a5eeece2f084bc5a5b91e03107eeee9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:36.026703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:36.057531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1051/2109] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:36.072672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.074096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:36.110475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1059/2110] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:36.111353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.183386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185752be-fbc4-48ed-8761-be72fee93b92 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.192326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.193918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.194222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.194506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.208766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.208766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.219954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1052/2111] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:09:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:09:36.225744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f9dd2-9867-49e3-93db-cfc9ae8543ee None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.232961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.233904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.234141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.234441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.235335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.237572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.299665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1060/2112] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:36 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:36.303809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.319926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f51a-89e0-4d1b-847b-5e2ad73e513d service swift] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.393568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.394356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.394723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.395069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.395875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.399379 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.429311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1053/2113] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:36 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:36.430845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.447341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3429c75-4276-43c0-bf7e-358e5e929022 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.460730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.462726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.463172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.463604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.465092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.467002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.485243 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1061/2114] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:36.486634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.501929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ce8e-afd5-4cd4-b720-6950755c0d73 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.522241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.523261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.523654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.524029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] PATH_INFO: `/v3/projects/52e8fbfbff004b9594a24cea887d210b/users/23aae6edb0cc4d2da2d1a01a92ce30a7/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.539380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:36.546708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e8fbfbff004b9594a24cea887d210b, user_id=23aae6edb0cc4d2da2d1a01a92ce30a7, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.550966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.590107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1054/2115] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:36 2022] PUT /identity/v3/projects/52e8fbfbff004b9594a24cea887d210b/users/23aae6edb0cc4d2da2d1a01a92ce30a7/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:36.590716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.604365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a215e-c346-433e-8891-313adb6d2a93 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.707160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.708649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.708941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.709438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.710230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.711937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f1f1e-f8d8-457d-a59e-a89ac2367696 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.721895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:36.729846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1062/2116] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:36.734482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:36.745462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d22fa-dd69-4641-9b96-34a657cf3728 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:36.750492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:36.751393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.751680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.751961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] PATH_INFO: `/v3/projects/52e8fbfbff004b9594a24cea887d210b/users/23aae6edb0cc4d2da2d1a01a92ce30a7/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.754078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e8fbfbff004b9594a24cea887d210b, user_id=23aae6edb0cc4d2da2d1a01a92ce30a7, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:36.756932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:36.780163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1055/2117] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:36 2022] PUT /identity/v3/projects/52e8fbfbff004b9594a24cea887d210b/users/23aae6edb0cc4d2da2d1a01a92ce30a7/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:36.793939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:36.793939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:36.793939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:36.830566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] MFA Rules not processed for user `23aae6edb0cc4d2da2d1a01a92ce30a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:36.831779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:37.010987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1063/2118] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:37.026998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:37.028799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:37.114321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567abde-6990-46bb-b932-9e368d52f0dc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:37.119592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:37.120923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:37.121224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:37.121513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:37.135730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:37.137191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66071e36-892e-4dd1-b038-9087b5a29307 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:37.151737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1056/2119] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:37.294232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:37.295998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:37.329435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36bc0f-3bb5-4c99-b1f7-ba7a315ca985 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:37.332571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:37.333346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:37.333699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:37.334054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:37.427516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:37.438113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:37.491183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1064/2120] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:09:46.640331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad579-f152-40d9-91de-9e664578dfb1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:46.640905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad579-f152-40d9-91de-9e664578dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:46.640905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad579-f152-40d9-91de-9e664578dfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:46.659107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c12ad579-f152-40d9-91de-9e664578dfb1 None None] MFA Rules not processed for user `1404717904a14000a11f06c6c8291105`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:46.660369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c12ad579-f152-40d9-91de-9e664578dfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:46.714445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1057/2121] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:47.438488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=62 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:55.334949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:55.397592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947e00-9619-431e-8dd9-5ce71a882dbd service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:55.402285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:55.403237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:55.403532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:55.404362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] PATH_INFO: `/v3/users/0a5eeece2f084bc5a5b91e03107eeee9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:55.405545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5eeece2f084bc5a5b91e03107eeee9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:55.407482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:55.455905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5eeece2f084bc5a5b91e03107eeee9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:55.462697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5eeece2f084bc5a5b91e03107eeee9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:55.463346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5eeece2f084bc5a5b91e03107eeee9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:55.470919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5eeece2f084bc5a5b91e03107eeee9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:55.483367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5eeece2f084bc5a5b91e03107eeee9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:55.489281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1065/2122] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:55 2022] DELETE /identity/v3/users/0a5eeece2f084bc5a5b91e03107eeee9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:55.493685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:55.494141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:55.494234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:55.511832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:55.513061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:55.572165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1058/2123] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:55.581813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:55.583290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:55.641407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eeffd1-ab86-416b-a2fc-3d14d08cc42b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:55.645532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:55.646186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:55.646475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:55.646765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:55.656078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:55.657553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:55.667349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1066/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:55.938224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:55.939880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:55.954706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc36226-fa13-4a98-ac6f-dd5db0819a96 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:55.962073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:55.963473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:55.963951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:55.964425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] PATH_INFO: `/v3/projects/8ab7d15c95b74d12b5e20d9bfcc9db67` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:55.966183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab7d15c95b74d12b5e20d9bfcc9db67)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:55.968813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:55.997387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:56.025798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab7d15c95b74d12b5e20d9bfcc9db67', 'request_id': 'req-77179f43-077b-4a8c-9e09-fd395e96e50e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.032611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab7d15c95b74d12b5e20d9bfcc9db67', 'request_id': 'req-77179f43-077b-4a8c-9e09-fd395e96e50e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.039079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab7d15c95b74d12b5e20d9bfcc9db67', 'request_id': 'req-77179f43-077b-4a8c-9e09-fd395e96e50e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.059948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] The token cache is being invalidate because project 8ab7d15c95b74d12b5e20d9bfcc9db67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:56.060314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.060945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.062491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1059/2125] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:55 2022] DELETE /identity/v3/projects/8ab7d15c95b74d12b5e20d9bfcc9db67 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:56.066727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:56.076455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809149f5-5e01-48a0-918e-a3efac5c09f7 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:56.095650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:56.136721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:56.137595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.137876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.138190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] PATH_INFO: `/v3/users/23aae6edb0cc4d2da2d1a01a92ce30a7` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.139381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23aae6edb0cc4d2da2d1a01a92ce30a7)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:56.141337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:56.188805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23aae6edb0cc4d2da2d1a01a92ce30a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.195662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23aae6edb0cc4d2da2d1a01a92ce30a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.196393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23aae6edb0cc4d2da2d1a01a92ce30a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.203959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23aae6edb0cc4d2da2d1a01a92ce30a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.215897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-809ac3e1-98ee-46fa-a7ea-9d3365bffec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23aae6edb0cc4d2da2d1a01a92ce30a7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.222351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1067/2126] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:56 2022] DELETE /identity/v3/users/23aae6edb0cc4d2da2d1a01a92ce30a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:56.606957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:56.608587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:56.621871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77179f43-077b-4a8c-9e09-fd395e96e50e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:56.684218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:56.685083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.685366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.685653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] PATH_INFO: `/v3/projects/52e8fbfbff004b9594a24cea887d210b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.686867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e8fbfbff004b9594a24cea887d210b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:56.688990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:56.738942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e8fbfbff004b9594a24cea887d210b', 'request_id': 'req-ffe13625-42c0-4b0f-80ab-645c232babd0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.746484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e8fbfbff004b9594a24cea887d210b', 'request_id': 'req-ffe13625-42c0-4b0f-80ab-645c232babd0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.754466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e8fbfbff004b9594a24cea887d210b', 'request_id': 'req-ffe13625-42c0-4b0f-80ab-645c232babd0'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.771345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] The token cache is being invalidate because project 52e8fbfbff004b9594a24cea887d210b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:56.771938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.772712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ffe13625-42c0-4b0f-80ab-645c232babd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:56.774689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1060/2127] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:56 2022] DELETE /identity/v3/projects/52e8fbfbff004b9594a24cea887d210b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:56.780552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.781123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.782207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.813177 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:56.814388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:56.878932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1068/2128] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:56.887712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.888881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.889939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.915951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:56.917322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:56.933746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1061/2129] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:56.938519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:56.948235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c39d01-0971-44b0-b8f0-c5b59e0f5182 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:56.953394 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:56.954045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.954327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.954610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.955376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:09:56.955686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:56.957327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:56.969067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1069/2130] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:09:56.973106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:56.982196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80c08-4cc0-4528-9aad-2d53d949fc5e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:56.986926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:56.987573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:56.987860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:56.988162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:56.988853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:56.990752 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.017981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1062/2131] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:56 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:57.022097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.033703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6c6e3-1721-4065-9a38-c39151beb9b7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.052225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:57.098249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.099300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.099593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.099878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.100634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.102697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.125954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1070/2132] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:57 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:57.129846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.141099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f4cc0-4577-4e6c-9a5f-8637ea3dcaf5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.146254 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.146819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.147108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.147391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.148262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.149904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.152378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:57.165508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1063/2133] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:57.167977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.181288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3add-d3d8-4b57-b4b3-80d16fb0833c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.186593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.187253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.187539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.187816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.188579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.190171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2d4d1-95c4-4c22-9e0e-1848a1621a8c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.200824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1071/2134] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:57.204919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.215483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816d9a8-54ca-4dad-bfca-8f60d4da2183 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.220446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.221443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.221801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.222195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] PATH_INFO: `/v3/projects/f2a88994aa474570b4eab99015fea772/users/108a7f13361e476ba984e672971f0b4f/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.238871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a88994aa474570b4eab99015fea772, user_id=108a7f13361e476ba984e672971f0b4f, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.241777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.256557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1064/2135] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:57 2022] PUT /identity/v3/projects/f2a88994aa474570b4eab99015fea772/users/108a7f13361e476ba984e672971f0b4f/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:57.267323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.267615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.267898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.297539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] MFA Rules not processed for user `108a7f13361e476ba984e672971f0b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:57.298945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:57.364592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1072/2136] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:57.369062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.431188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27187-d3fb-4a0e-800f-bde98dae361a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.437285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.437951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.438260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.438530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.439668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.441802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.474156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1065/2137] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:09:57 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:57.478405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.498968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58843dea-ac22-498a-9f46-5b9532a4396c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.571016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.571782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.571975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.572329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.573375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.575423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.604398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1073/2138] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:09:57 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:09:57.605990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.619298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b3aba-aaa5-4c7f-9b59-7bad0f27d5cf admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.624575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.625597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.625956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.628859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.628859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.628859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.644455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1066/2139] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:09:57.644972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.655963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80056ed8-6d97-4c1d-8ba8-e34a5b57a7aa admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.660604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.661771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.661771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.662056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] PATH_INFO: `/v3/projects/565142bbd83e4efd87464c1318e910c6/users/09419cd07afe4391a20b04abab6ee381/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.683916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565142bbd83e4efd87464c1318e910c6, user_id=09419cd07afe4391a20b04abab6ee381, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.686588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.691616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:57.717179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1074/2140] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:57 2022] PUT /identity/v3/projects/565142bbd83e4efd87464c1318e910c6/users/09419cd07afe4391a20b04abab6ee381/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:09:57.721124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.732999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f762e-705d-4c10-8e53-1fbc470773ae admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.791709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.792412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.792697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.793040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.793858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.796139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dffa6d4-3029-4d03-878d-1f84dcad7b6c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.810003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1067/2141] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:09:57.811295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.821661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841751f-ebdc-408a-b434-c352569818c2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:57.826311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:57.827187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.827475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.827751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] PATH_INFO: `/v3/projects/565142bbd83e4efd87464c1318e910c6/users/09419cd07afe4391a20b04abab6ee381/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.829783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=565142bbd83e4efd87464c1318e910c6, user_id=09419cd07afe4391a20b04abab6ee381, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:57.832454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:57.843247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1075/2142] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:09:57 2022] PUT /identity/v3/projects/565142bbd83e4efd87464c1318e910c6/users/09419cd07afe4391a20b04abab6ee381/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:57.854467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:57.854759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:57.855048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:57.881803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] MFA Rules not processed for user `09419cd07afe4391a20b04abab6ee381`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:57.882941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:57.919186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:57.985225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1068/2143] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:57.994652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:57.999825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:58.077928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1666004-201f-4515-91d0-ccaf7c2f0c10 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:58.083480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:58.084462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:58.084828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:58.085213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:58.098436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:58.100231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72075f2d-38c0-4ddd-a568-8e9cc92195bf service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:58.136253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1076/2144] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:58.792033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:58.871195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f60d34-a283-4c6a-922a-f7187a5dad45 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:58.876554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:58.877448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:58.877733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:58.877997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] PATH_INFO: `/v3/users/108a7f13361e476ba984e672971f0b4f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:58.879709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] RBAC: Authorizing `identity:delete_user(user_id=108a7f13361e476ba984e672971f0b4f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:58.881920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:58.940184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108a7f13361e476ba984e672971f0b4f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:58.941215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108a7f13361e476ba984e672971f0b4f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:58.941898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108a7f13361e476ba984e672971f0b4f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:58.950890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108a7f13361e476ba984e672971f0b4f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:58.963885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108a7f13361e476ba984e672971f0b4f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:58.977466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1069/2145] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:58 2022] DELETE /identity/v3/users/108a7f13361e476ba984e672971f0b4f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:58.982661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:58.983474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:58.983765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:59.002941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:09:59.004193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:59.086344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1077/2146] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:09:59.097727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:59.099363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:59.184462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:09:59.189515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77741a-5b26-43db-b118-7130d3c513de admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:59.197446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:59.198308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:59.198644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:59.199000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:59.210160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:59.211626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:59.222928 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1070/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:09:59.577553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:59.579023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:09:59.593568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8b1be-a3b5-449d-a977-13cbad299a55 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:59.598492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:59.599599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:59.600100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:59.600514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] PATH_INFO: `/v3/projects/f2a88994aa474570b4eab99015fea772` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:59.601807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a88994aa474570b4eab99015fea772)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:59.604096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:59.652431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a88994aa474570b4eab99015fea772', 'request_id': 'req-043b7df1-e0ea-4d98-a85c-7c8307295966'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.660439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a88994aa474570b4eab99015fea772', 'request_id': 'req-043b7df1-e0ea-4d98-a85c-7c8307295966'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.666752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a88994aa474570b4eab99015fea772', 'request_id': 'req-043b7df1-e0ea-4d98-a85c-7c8307295966'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.677638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] The token cache is being invalidate because project f2a88994aa474570b4eab99015fea772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:09:59.677932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.678565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.680318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1078/2148] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:09:59 2022] DELETE /identity/v3/projects/f2a88994aa474570b4eab99015fea772 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:09:59.684322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:09:59.693841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997f60-3d81-4deb-90bb-3315943d2252 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:09:59.750671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:09:59.751536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:09:59.751817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:09:59.752090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] PATH_INFO: `/v3/users/09419cd07afe4391a20b04abab6ee381` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:09:59.753357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09419cd07afe4391a20b04abab6ee381)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:09:59.755276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:09:59.802742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09419cd07afe4391a20b04abab6ee381'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.803427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09419cd07afe4391a20b04abab6ee381'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.804002 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09419cd07afe4391a20b04abab6ee381'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.811912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09419cd07afe4391a20b04abab6ee381'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.826739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-996b8339-a833-40e8-890e-33da26063a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09419cd07afe4391a20b04abab6ee381'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:09:59.839307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1071/2149] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:09:59 2022] DELETE /identity/v3/users/09419cd07afe4391a20b04abab6ee381 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:00.195763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.197286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:00.211307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043b7df1-e0ea-4d98-a85c-7c8307295966 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.215771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:00.281228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.282090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.282389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.282652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] PATH_INFO: `/v3/projects/565142bbd83e4efd87464c1318e910c6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.283856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=565142bbd83e4efd87464c1318e910c6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.285812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.332920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565142bbd83e4efd87464c1318e910c6', 'request_id': 'req-0667f451-7b61-4de6-ab30-a9d6b6145fd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:00.340160 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565142bbd83e4efd87464c1318e910c6', 'request_id': 'req-0667f451-7b61-4de6-ab30-a9d6b6145fd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:00.345930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565142bbd83e4efd87464c1318e910c6', 'request_id': 'req-0667f451-7b61-4de6-ab30-a9d6b6145fd9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:00.357256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] The token cache is being invalidate because project 565142bbd83e4efd87464c1318e910c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:00.357546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:00.358171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0667f451-7b61-4de6-ab30-a9d6b6145fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:00.359871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1079/2150] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:00 2022] DELETE /identity/v3/projects/565142bbd83e4efd87464c1318e910c6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:00.365041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.365534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.365827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.385757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:00.386954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:00.450355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1072/2151] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:00.458126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.458455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.458741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.478978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:00.480372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:00.491995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1080/2152] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:00.496521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.509629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312cdda-7fd8-43e4-9bf3-4d4880955007 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.516122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.516413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.517051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.517401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.518222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:10:00.518595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.521469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.530024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:00.538533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1073/2153] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:10:00.543310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.563082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1e7c6-75f7-485d-9e4a-9688c2b9b39e None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.580872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.581884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.583490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.583881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.585633 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.590641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.618466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.621667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:00.641054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1081/2154] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:00 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:00.655017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.751441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66482e-d88d-4d48-a175-8fb19d138018 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.772159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54482b2-9999-450b-abc1-bd4c391a9fab admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.778234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.779894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.780425 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.781008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.782120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.784527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.819829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:00.830042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1082/2155] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:00 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:00.834251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.849012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03e479-ac89-480b-a035-6358ef471a14 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.854960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.855501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.855984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.856123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.856404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.856686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.857665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.858042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.858158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.859926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.861276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:00.879203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1083/2156] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:00.880384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.882183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.883973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.896384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e3cde-979e-440a-bdce-d3d16508bf9e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.902322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1074/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:10:00.905565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.906259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.906553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.906835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:00.907866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:00.909921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303bef55-04ed-4172-9a74-dbd5f40fc760 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:00.939219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1084/2158] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:00.939988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:00.960369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126e647-5a37-47b6-9642-a9f2294d7998 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:00.990356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:00.996593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:00.996593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:00.998107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] PATH_INFO: `/v3/projects/3822e3b85ae94a0990e9660e189cc287/users/675441849d4f4c028a8087bf39ecc504/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.039095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3822e3b85ae94a0990e9660e189cc287, user_id=675441849d4f4c028a8087bf39ecc504, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.044984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.067100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1075/2159] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:00 2022] PUT /identity/v3/projects/3822e3b85ae94a0990e9660e189cc287/users/675441849d4f4c028a8087bf39ecc504/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:10:01.081076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.082347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.082743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.139614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] MFA Rules not processed for user `675441849d4f4c028a8087bf39ecc504`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:01.140931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:01.300947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1085/2160] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:01.306037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:01.392515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:01.416951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9dcf20-7a6a-4dc1-98d8-e37e78451e4f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:01.425827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:01.426871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.427603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.428155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.429884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.435369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.478790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1076/2161] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:01.483231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:01.496144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39fce80-bb80-4e94-881b-4bdae10baec0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:01.592934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:01.593854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.594422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.594685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.595516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.598297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.636913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1086/2162] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:01 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:01.641636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:01.655546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede17f1-a7c7-42ff-9ca6-cbb97fbdf808 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:01.662023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:01.662877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.663250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.663651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.664858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.667058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.679680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1077/2163] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:01.683818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:01.695130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4153fa-3583-4f4f-bc22-e87c11de33f2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:01.700397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:01.701663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.701953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.702263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.707849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.707849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0f093-2b81-4fa4-91e6-4ccb02018a0b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.716915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1087/2164] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:01.721871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:01.743396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30033f-abdc-4af5-b711-1f3ff3c077e1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:01.758745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:01.759731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.760029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.760338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] PATH_INFO: `/v3/projects/86e8a3a8263246ca9eeb34b6edcbabd0/users/30799b6d11e542bcbbe5936617792739/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.796161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e8a3a8263246ca9eeb34b6edcbabd0, user_id=30799b6d11e542bcbbe5936617792739, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:01.798959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:01.824978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1078/2165] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:01 2022] PUT /identity/v3/projects/86e8a3a8263246ca9eeb34b6edcbabd0/users/30799b6d11e542bcbbe5936617792739/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:01.829510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:01.830120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:01.830441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:01.869692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] MFA Rules not processed for user `30799b6d11e542bcbbe5936617792739`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:01.871548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:01.912809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:01.975331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1088/2166] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:01.979859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.085326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef5a19-ed77-46f2-92d8-68f64593eaaa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.094502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.096164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.096564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.097049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.097932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.100396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.143050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1079/2167] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:10:02.147119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.160824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8285e3-3c18-46aa-998f-69f3b8ab178f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.231094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.232023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.232435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.232809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.233923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.235986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.268329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1089/2168] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:02 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:02.269292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.283005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8d24f-e276-4dab-a228-046e67fe82cb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.292858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.292858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.292858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.292858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.294829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.294829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.308524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1080/2169] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:02.313327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.323826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090b77c-7843-41d6-bf5b-84e0ed712fe2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.331527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.333102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.333608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.334346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] PATH_INFO: `/v3/projects/b906739abffe4a11a99253d12625edc1/users/f9bb1e7510aa4a9baca95ed11a75d761/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.356992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b906739abffe4a11a99253d12625edc1, user_id=f9bb1e7510aa4a9baca95ed11a75d761, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.360051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.376099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1090/2170] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:02 2022] PUT /identity/v3/projects/b906739abffe4a11a99253d12625edc1/users/f9bb1e7510aa4a9baca95ed11a75d761/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:02.379847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.390679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df0f85-3377-4409-84c8-8c7b8c2a21da admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.456022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.456795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.457147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.457493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.458229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.459878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468417b6-e5c3-4391-98d8-0b362fffd900 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.472272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1081/2171] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:02.474156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.486779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeeee23-a966-4494-8cd7-deb281ce9011 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.490913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.491929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.492369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.492699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] PATH_INFO: `/v3/projects/b906739abffe4a11a99253d12625edc1/users/f9bb1e7510aa4a9baca95ed11a75d761/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.494826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b906739abffe4a11a99253d12625edc1, user_id=f9bb1e7510aa4a9baca95ed11a75d761, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.498292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d0bdc-43e8-4cbd-be7e-3a77d347c2b7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.509297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1091/2172] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:02 2022] PUT /identity/v3/projects/b906739abffe4a11a99253d12625edc1/users/f9bb1e7510aa4a9baca95ed11a75d761/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:02.519686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.519979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.520293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.533010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:02.551605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] MFA Rules not processed for user `f9bb1e7510aa4a9baca95ed11a75d761`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:02.553734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:02.679933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1082/2173] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:02.693115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e09e-1be4-4c96-a658-a12d7123e31b None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.693411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e09e-1be4-4c96-a658-a12d7123e31b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.693694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3e09e-1be4-4c96-a658-a12d7123e31b None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.695138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1092/2174] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:10:02.698959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.700527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:02.705754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733b5d-0733-414a-8e8e-9f8810749aa8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.706043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733b5d-0733-414a-8e8e-9f8810749aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.706325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2733b5d-0733-414a-8e8e-9f8810749aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.787030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b2733b5d-0733-414a-8e8e-9f8810749aa8 None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:02.788593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b2733b5d-0733-414a-8e8e-9f8810749aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:02.842527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a35a86-d38a-4e97-ba65-e383aa416298 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:02.852534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:02.853192 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:02.853647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:02.853929 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:02.872197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:02.881004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:02.882921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:02.908513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1083/2175] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:02.958362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1093/2176] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:10:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:10:02.973054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:02.974734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.065542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c60ed-1308-48d8-b38d-4f552e414c1d service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:03.069839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:03.070491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.070788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.071087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.085031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:03.086786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50006e7d-1cb3-4246-935b-166b342f463e service glance] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:03.100819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1084/2177] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:03.173456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087890fb-562a-47ce-b914-4ba434970635 None None] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.173761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087890fb-562a-47ce-b914-4ba434970635 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.174028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087890fb-562a-47ce-b914-4ba434970635 None None] PATH_INFO: `/` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.175906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1094/2178] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:10:03.185233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464663-99f5-410f-a311-a89b95fbc73d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.185786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464663-99f5-410f-a311-a89b95fbc73d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.186463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e464663-99f5-410f-a311-a89b95fbc73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.214740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0e464663-99f5-410f-a311-a89b95fbc73d None None] MFA Rules not processed for user `014dfb1baeb24fa7ab38923a6962db4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:03.216659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0e464663-99f5-410f-a311-a89b95fbc73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.230529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:03.275769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1085/2179] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:10:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:03.288428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087890fb-562a-47ce-b914-4ba434970635 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:03.290523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-087890fb-562a-47ce-b914-4ba434970635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.304349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087890fb-562a-47ce-b914-4ba434970635 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:03.307880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:03.308548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.308835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.309114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.321918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:03.323391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a6d842-3eb6-4001-b324-a4001e17b874 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:03.332215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1095/2180] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:03.422962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.423262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.423540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] PATH_INFO: `/` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.425056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1086/2181] 10.222.0.23 () {54 vars in 921 bytes} [Mon Jun 20 07:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 07:10:03.430887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.431183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.431463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.451163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] MFA Rules not processed for user `e19cf623e22c485fb1bb8f0ada7114b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:03.452423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.462254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1096/2182] 10.222.0.23 () {58 vars in 1018 bytes} [Mon Jun 20 07:10:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:10:03.473855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:03.474010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:03.475692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.476335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:03.490099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c67b4-f6fd-431b-bb8d-57ba452a0f37 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:03.490616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8b11a-d4db-4314-9af6-c0514432feda None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:03.494306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:03.494487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:03.495084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.495195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:03.495438 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.495537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:03.495781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.495887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:03.509453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:03.511075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:03.541553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1097/2183] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:03.577824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:03.579866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:03.589312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1087/2184] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:07.654244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:07.663445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b941087-83c6-4911-9c46-b025f637a0b8 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:07.667102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:07.667691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:07.667963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:07.668278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:07.725276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:07.726768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b4790-ceb5-4d03-ba57-c818c8ecd96e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:07.736209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1098/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:09.884044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:09.948505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f45ad-da07-4997-b61c-8f33a829d90b service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:09.954297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:09.955484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:09.955772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:09.956040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] PATH_INFO: `/v3/users/675441849d4f4c028a8087bf39ecc504` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:09.957321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675441849d4f4c028a8087bf39ecc504)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:09.959302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:10.009409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675441849d4f4c028a8087bf39ecc504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.010073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675441849d4f4c028a8087bf39ecc504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.010673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675441849d4f4c028a8087bf39ecc504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.018355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675441849d4f4c028a8087bf39ecc504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.029781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:10.033258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675441849d4f4c028a8087bf39ecc504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.047938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1088/2186] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:09 2022] DELETE /identity/v3/users/675441849d4f4c028a8087bf39ecc504 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:10.052417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:10.053248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:10.053530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:10.070431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:10.071631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:10.131754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1099/2187] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:10.142424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:10.143827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:10.202032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686ff6a-4bc0-4821-9aa8-82ab3300c443 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:10.205559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:10.206206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:10.206494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:10.206770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:10.215940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:10.218350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:10.228203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1089/2188] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:10.522788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:10.525240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:10.572138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189d634-9e1e-434a-8afe-cdb650ec4cb6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:10.582581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:10.584043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:10.584368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:10.584787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] PATH_INFO: `/v3/projects/3822e3b85ae94a0990e9660e189cc287` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:10.585956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3822e3b85ae94a0990e9660e189cc287)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:10.588423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:10.636010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:10.651468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3822e3b85ae94a0990e9660e189cc287', 'request_id': 'req-bbccadd5-8357-428e-b03b-e0c86bdd8883'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.661763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3822e3b85ae94a0990e9660e189cc287', 'request_id': 'req-bbccadd5-8357-428e-b03b-e0c86bdd8883'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.684097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3822e3b85ae94a0990e9660e189cc287', 'request_id': 'req-bbccadd5-8357-428e-b03b-e0c86bdd8883'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.714471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] The token cache is being invalidate because project 3822e3b85ae94a0990e9660e189cc287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:10.714757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.715475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.720918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1100/2189] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:10 2022] DELETE /identity/v3/projects/3822e3b85ae94a0990e9660e189cc287 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:10.722196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:10.733071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711d77e-efb3-49ca-b2fa-db0aee9048c5 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:10.798383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:10.798839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:10.799195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:10.799524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] PATH_INFO: `/v3/users/30799b6d11e542bcbbe5936617792739` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:10.800971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=30799b6d11e542bcbbe5936617792739)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:10.803221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:10.855714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30799b6d11e542bcbbe5936617792739'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.856822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30799b6d11e542bcbbe5936617792739'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.857695 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30799b6d11e542bcbbe5936617792739'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.865624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30799b6d11e542bcbbe5936617792739'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.878460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97a81ac2-391d-4bb7-a457-2f9f7e160e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30799b6d11e542bcbbe5936617792739'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:10.892760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1090/2190] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:10 2022] DELETE /identity/v3/users/30799b6d11e542bcbbe5936617792739 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:11.192603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:11.194284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:11.220045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccadd5-8357-428e-b03b-e0c86bdd8883 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:11.290559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:11.290876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:11.291377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:11.309327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:11.332376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:11.332611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:11.333667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:11.334127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:11.334432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:11.334747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] PATH_INFO: `/v3/projects/86e8a3a8263246ca9eeb34b6edcbabd0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:11.336255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e8a3a8263246ca9eeb34b6edcbabd0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:11.339338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:11.393214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e8a3a8263246ca9eeb34b6edcbabd0', 'request_id': 'req-47377b04-b8ec-4afb-941f-d23b67c23d49'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.405789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e8a3a8263246ca9eeb34b6edcbabd0', 'request_id': 'req-47377b04-b8ec-4afb-941f-d23b67c23d49'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.410495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1091/2191] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:10:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:11.411844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e8a3a8263246ca9eeb34b6edcbabd0', 'request_id': 'req-47377b04-b8ec-4afb-941f-d23b67c23d49'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.425135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] The token cache is being invalidate because project 86e8a3a8263246ca9eeb34b6edcbabd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:11.425238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.425857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.427545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1101/2192] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:11 2022] DELETE /identity/v3/projects/86e8a3a8263246ca9eeb34b6edcbabd0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:11.431546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:11.432874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:11.433327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:11.434183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:11.447931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30007d-8aa4-402c-a6d6-97705f3d09fc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:11.484082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:11.507383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47377b04-b8ec-4afb-941f-d23b67c23d49 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:11.507871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:11.508784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:11.509070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:11.509349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] PATH_INFO: `/v3/users/f9bb1e7510aa4a9baca95ed11a75d761` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:11.510756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bb1e7510aa4a9baca95ed11a75d761)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:11.512731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:11.513091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:11.513750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:11.514048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:11.514350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:11.516280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:11.566986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bb1e7510aa4a9baca95ed11a75d761'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.567583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bb1e7510aa4a9baca95ed11a75d761'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.568299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bb1e7510aa4a9baca95ed11a75d761'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.576883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bb1e7510aa4a9baca95ed11a75d761'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.577308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:11.579009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:11.588998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1102/2193] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 07:10:11.591033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-2bff8213-df2c-4956-9ecb-802f1f4860f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bb1e7510aa4a9baca95ed11a75d761'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:11.605780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1092/2194] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:11 2022] DELETE /identity/v3/users/f9bb1e7510aa4a9baca95ed11a75d761 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:12.103391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.105107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:12.152659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e972a23-1323-454d-bff6-4aba0157a0a1 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.241462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.244037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.244524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.244960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] PATH_INFO: `/v3/projects/b906739abffe4a11a99253d12625edc1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.246417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b906739abffe4a11a99253d12625edc1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.248758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.305690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b906739abffe4a11a99253d12625edc1', 'request_id': 'req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:12.315098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b906739abffe4a11a99253d12625edc1', 'request_id': 'req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:12.320783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b906739abffe4a11a99253d12625edc1', 'request_id': 'req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:12.332494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] The token cache is being invalidate because project b906739abffe4a11a99253d12625edc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:12.332879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:12.333635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ee01d56a-fe4e-4234-8167-0b09a07f6dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:12.335379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1103/2195] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:12 2022] DELETE /identity/v3/projects/b906739abffe4a11a99253d12625edc1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:12.342309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3195a-e407-4234-aee8-217497365061 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.343380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3195a-e407-4234-aee8-217497365061 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.343478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3195a-e407-4234-aee8-217497365061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.362583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-58c3195a-e407-4234-aee8-217497365061 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:12.364145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-58c3195a-e407-4234-aee8-217497365061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:12.392360 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:12.452399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1093/2196] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:12.457463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.457657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.457940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.478888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:12.480143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:12.489081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1104/2197] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:10:12.493102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3195a-e407-4234-aee8-217497365061 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.504582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3195a-e407-4234-aee8-217497365061 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.509800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.510550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.510896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.511186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.511935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:10:12.512349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.514057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.527985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1094/2198] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:10:12.532515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.542498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2931b-50a0-4a8e-b863-d54d941796d3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.551594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.552437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.552793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.553141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.553922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.556665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.579514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:12.608667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1105/2199] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/projects => generated 401 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:10:12.609105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.622259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1199-0112-4707-854b-107ec9a38084 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.693881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.694668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.695051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.695404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.696247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.698241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.722581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1095/2200] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:12.726306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.736721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af5598-9817-4b89-8613-cbc53177966e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.741628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.742287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.742572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.742860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.743727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.745376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.760038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1106/2201] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:12.761441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.774917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd45987-ad65-455c-a346-3ab8461a80e7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.779762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.780465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.780756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.781033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.781772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.783389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6610c215-ebf4-43f6-83e1-72e37108761e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.796500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1096/2202] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:12.798018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:12.808518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e18fa-90fd-4e80-9a89-8428c586105f admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:12.813852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:12.814744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.815027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.815305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] PATH_INFO: `/v3/projects/28967532a9d9460587a9bc586047e343/users/1e6fe088640c4b9da1454564c3290df9/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.832298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28967532a9d9460587a9bc586047e343, user_id=1e6fe088640c4b9da1454564c3290df9, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:12.834998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:12.847847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1107/2203] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:12 2022] PUT /identity/v3/projects/28967532a9d9460587a9bc586047e343/users/1e6fe088640c4b9da1454564c3290df9/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:12.860061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:12.860428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:12.860721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:12.890651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] MFA Rules not processed for user `1e6fe088640c4b9da1454564c3290df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:12.892183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:12.961713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1097/2204] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:12.962845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.029538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1acd78-1c86-4954-a9d1-a1a471e52556 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.034567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.035381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.035804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.036087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.036818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.038741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.066135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1108/2205] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:12 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:10:13.070088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.082465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bf523-b80d-41ce-a7d4-24954ba08852 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.109256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:13.146909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.147779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.148067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.148391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.149106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.151023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.176367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1098/2206] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:13 2022] POST /identity/v3/users => generated 307 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:13.177763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.190221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494236-f851-4b5e-b907-b26f6a3e6905 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.195532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.196503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.196992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.197332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.198105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.199722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.211099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1109/2207] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:13.215419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.228645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376032c-8305-4a93-bf26-23b52a9f6d02 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.234484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.235382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.235669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.235951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] PATH_INFO: `/v3/projects/d0d89355d9534870bc21a196d84eb1ea/users/61a7ee406e2e4ab49014c749c7061cbc/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.259436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d89355d9534870bc21a196d84eb1ea, user_id=61a7ee406e2e4ab49014c749c7061cbc, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.262218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.278799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1099/2208] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:13 2022] PUT /identity/v3/projects/d0d89355d9534870bc21a196d84eb1ea/users/61a7ee406e2e4ab49014c749c7061cbc/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:13.280527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.293848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866d8eb-a91f-4445-a25d-c73d4b625db5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.381373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.382145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.382538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.382979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.383821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.385583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a31706-5853-49ec-a20c-32b39162d5ba admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.399931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1110/2209] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:13.401149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.414470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958260b8-db14-4e5b-b04d-3437391b95aa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.419843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.421452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.422820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.422820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] PATH_INFO: `/v3/projects/d0d89355d9534870bc21a196d84eb1ea/users/61a7ee406e2e4ab49014c749c7061cbc/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.425423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0d89355d9534870bc21a196d84eb1ea, user_id=61a7ee406e2e4ab49014c749c7061cbc, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.428218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.440941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1100/2210] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:13 2022] PUT /identity/v3/projects/d0d89355d9534870bc21a196d84eb1ea/users/61a7ee406e2e4ab49014c749c7061cbc/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:13.452058 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.456164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.456164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.501531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] MFA Rules not processed for user `61a7ee406e2e4ab49014c749c7061cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:13.502842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:13.518766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:13.592865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1111/2211] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:13 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:13.605306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.607110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:13.668723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847dde80-83d1-4c4f-80fd-4338a2c77f31 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.672657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.673471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.673822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.674266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.684138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.685596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.696175 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1101/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:13 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:13.770430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:13.780813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c896f8b-a5d0-40c1-a744-75d882aa83c8 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:13.784507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:13.785307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:13.785605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:13.785881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:13.845039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:13.846495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdedea-8943-4236-abc1-0ad489b28f72 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:13.857089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1112/2213] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:19.099699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:19.174662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fec9d-3179-4943-92b4-781a2f49318f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:19.179202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:19.180063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:19.180398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:19.180662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] PATH_INFO: `/v3/users/1e6fe088640c4b9da1454564c3290df9` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:19.181994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6fe088640c4b9da1454564c3290df9)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:19.183951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:19.207419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:19.234646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6fe088640c4b9da1454564c3290df9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.235311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6fe088640c4b9da1454564c3290df9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.235926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6fe088640c4b9da1454564c3290df9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.242949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6fe088640c4b9da1454564c3290df9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.255228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6fe088640c4b9da1454564c3290df9'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.267903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1102/2214] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:19 2022] DELETE /identity/v3/users/1e6fe088640c4b9da1454564c3290df9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:19.272327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:19.272851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:19.272955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:19.289608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:19.290865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:19.353586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1113/2215] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:19.363422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:19.365053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:19.431649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93891f3a-c676-43dd-9657-4ac25cd6610f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:19.435398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:19.436150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:19.436445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:19.436721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:19.446738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:19.448222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:19.459071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1103/2216] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:19.833969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:19.835440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:19.848399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047de94c-b81e-405c-ad9f-10cb483edb08 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:19.860628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:19.861980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:19.862334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:19.862687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] PATH_INFO: `/v3/projects/28967532a9d9460587a9bc586047e343` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:19.864151 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=28967532a9d9460587a9bc586047e343)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:19.866256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:19.914465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:19.917806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28967532a9d9460587a9bc586047e343', 'request_id': 'req-3fae20fe-2730-4c30-b2ff-d903058205eb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.925232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28967532a9d9460587a9bc586047e343', 'request_id': 'req-3fae20fe-2730-4c30-b2ff-d903058205eb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.931617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28967532a9d9460587a9bc586047e343', 'request_id': 'req-3fae20fe-2730-4c30-b2ff-d903058205eb'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.943710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] The token cache is being invalidate because project 28967532a9d9460587a9bc586047e343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:19.944144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.944875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:19.949448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1114/2217] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:19 2022] DELETE /identity/v3/projects/28967532a9d9460587a9bc586047e343 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:19.950577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:19.961813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7622df-f80b-484f-900c-5ec7abb4986a service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:20.030447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:20.031336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.031640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.031919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] PATH_INFO: `/v3/users/61a7ee406e2e4ab49014c749c7061cbc` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.033211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a7ee406e2e4ab49014c749c7061cbc)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:20.035190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:20.085375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:20.087630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a7ee406e2e4ab49014c749c7061cbc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.093826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a7ee406e2e4ab49014c749c7061cbc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.095956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a7ee406e2e4ab49014c749c7061cbc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.127341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a7ee406e2e4ab49014c749c7061cbc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.144098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0e3ace09-d996-4579-8b0a-3f70f457d2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a7ee406e2e4ab49014c749c7061cbc'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.157302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1104/2218] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:19 2022] DELETE /identity/v3/users/61a7ee406e2e4ab49014c749c7061cbc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:20.486483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:20.488028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:20.505293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae20fe-2730-4c30-b2ff-d903058205eb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:20.578850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:20.579845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.580259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.580677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] PATH_INFO: `/v3/projects/d0d89355d9534870bc21a196d84eb1ea` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.581977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0d89355d9534870bc21a196d84eb1ea)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:20.583993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:20.638751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d89355d9534870bc21a196d84eb1ea', 'request_id': 'req-83132f5e-70c2-4736-a49d-bfde2a9043aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.646260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d89355d9534870bc21a196d84eb1ea', 'request_id': 'req-83132f5e-70c2-4736-a49d-bfde2a9043aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.651781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d89355d9534870bc21a196d84eb1ea', 'request_id': 'req-83132f5e-70c2-4736-a49d-bfde2a9043aa'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.662642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] The token cache is being invalidate because project d0d89355d9534870bc21a196d84eb1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:20.662934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.663562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-83132f5e-70c2-4736-a49d-bfde2a9043aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:20.665374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1115/2219] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:20 2022] DELETE /identity/v3/projects/d0d89355d9534870bc21a196d84eb1ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:10:20.669969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.670862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.671214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.690353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:20.696425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:20.755444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1105/2220] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:20.763019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.763310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.763593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.786825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:20.788091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:20.804027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1116/2221] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:10:20.809028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:20.841839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02a7cc-a6a3-4074-9a71-3f6cbd8a6811 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:20.847422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:20.848119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.848433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.848719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.849469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:10:20.849784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:20.851413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:20.877960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1106/2222] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:10:20.879266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:20.889632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae8304-d4ee-4439-8958-5a1bf33ad9c7 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:20.906053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:20.906732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:20.907016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:20.907291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:20.908000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:20.909957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:20.961825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1117/2223] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:20 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:20.963025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:20.996897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cc17-261d-4c4f-8b12-3c1e3b310c7b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.076409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.077400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.078082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.078268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.079630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.081854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.123018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1107/2224] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:20 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:21.125100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.155358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7c10a9-38cd-4844-a044-fc90824696b3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.168713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.169495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.170731 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.171038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.172602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.175766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.187130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1118/2225] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:21.191118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.201259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd459eba-3f4e-400e-a77e-799f9f6bfb0a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.206426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.207075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.207364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.207645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.208406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.210005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af009a-295f-47e3-ba98-8357924249a0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.220678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1108/2226] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:21.224466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.234324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa5062-9807-4695-9d29-238b256b7d45 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.238834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.239693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.239975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.240291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] PATH_INFO: `/v3/projects/2e353963318c43b698d5d45c01a869f5/users/0d7ed29ab94849a7a998d2cd957ae74f/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.254478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e353963318c43b698d5d45c01a869f5, user_id=0d7ed29ab94849a7a998d2cd957ae74f, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.257169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.265832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:21.271172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1119/2227] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:21 2022] PUT /identity/v3/projects/2e353963318c43b698d5d45c01a869f5/users/0d7ed29ab94849a7a998d2cd957ae74f/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:21.281871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.282165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.282448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.311976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] MFA Rules not processed for user `0d7ed29ab94849a7a998d2cd957ae74f`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:21.313244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:21.334968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:21.388448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1109/2228] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:21.393920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.460717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac08f-1811-4e49-b4fb-6034d3574d58 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.465890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.466569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.466864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.467700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.468472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.470430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.504716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1120/2229] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:10:21.506071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.518724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a64ac8-a808-4933-b43c-8a5c872502f8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.588451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.589386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.589758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.590104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.591047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.593759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.625565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1110/2230] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:10:21.629162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.644198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d70b30-22d7-4e39-b12a-550dee71e84e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.651062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.651062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.651062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.651062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.652894 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.656150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.664559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1121/2231] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:10:21.668340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.679262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad8b9d-22af-400b-9675-8c5e9e6a295e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.683996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.685136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.685136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.685326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.686486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.688277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253c4a34-e1f6-4cec-8b7b-0ef48f7ce7e3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.698922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1111/2232] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:21.703643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.715811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f4149-5c6c-45f2-84df-f9cbd58c4a45 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.721700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.722581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.722971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.723263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] PATH_INFO: `/v3/projects/4df990044bcc4efe801a6741c586e5b1/users/b76fb034ea324fb1b98a5b1e60c76c50/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.739740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df990044bcc4efe801a6741c586e5b1, user_id=b76fb034ea324fb1b98a5b1e60c76c50, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.743740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.761360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1122/2233] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:21 2022] PUT /identity/v3/projects/4df990044bcc4efe801a6741c586e5b1/users/b76fb034ea324fb1b98a5b1e60c76c50/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:21.774779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.775173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.775530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.803244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] MFA Rules not processed for user `b76fb034ea324fb1b98a5b1e60c76c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:21.804491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:21.871992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1112/2234] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:21.877106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.942406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7bf7-2be8-4609-a861-ee97b7e1963b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:21.947028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:21.947682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:21.947966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:21.948273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:21.948979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:21.950917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:21.976987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1123/2235] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:21.980840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:21.992299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a418-af56-4afc-aae5-e991ccdfdc8e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.016216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:22.048766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.049424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.049707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.049981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.050663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.052609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.071174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1113/2236] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:10:21 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:10:22.075058 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:22.085424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478c01-0d84-4bf1-b6c7-bf66f40dd852 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.090073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.090726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.091012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.091292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.092015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.093652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.103929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1124/2237] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:10:22.108280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:22.118886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67c6ce-ddb8-4cd3-9cd8-75356e9bd47f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.128248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.129143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.129431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.129708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] PATH_INFO: `/v3/projects/2780f57bad34411dbaa9031073d0f4cf/users/dfee7544125e456e810db9fa5f34372c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.147109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2780f57bad34411dbaa9031073d0f4cf, user_id=dfee7544125e456e810db9fa5f34372c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.150098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.186566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1114/2238] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:22 2022] PUT /identity/v3/projects/2780f57bad34411dbaa9031073d0f4cf/users/dfee7544125e456e810db9fa5f34372c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:22.187483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:22.199766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1391f7a-b28f-4f42-ac60-d29ff36cde8a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.225476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:22.263405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.264100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.264403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.264682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.265413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.267283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f55db-3a60-4a4c-9a51-6b00a925e95f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.280977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1125/2239] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:10:22.282199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:22.292470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135db10-2bf2-45b5-9356-747c2cba9778 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.297786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.298673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.298956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.299240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] PATH_INFO: `/v3/projects/2780f57bad34411dbaa9031073d0f4cf/users/dfee7544125e456e810db9fa5f34372c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.301269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2780f57bad34411dbaa9031073d0f4cf, user_id=dfee7544125e456e810db9fa5f34372c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.303950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30f8b8-dd14-4ae9-a3f0-ab6c2a036ef8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.316148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1115/2240] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:10:22 2022] PUT /identity/v3/projects/2780f57bad34411dbaa9031073d0f4cf/users/dfee7544125e456e810db9fa5f34372c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:22.325670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.325967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.326248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.343135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.343430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.343713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.362608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] MFA Rules not processed for user `0dbf681122194384812c2eb96d1adfbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:22.364512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:22.364737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] MFA Rules not processed for user `dfee7544125e456e810db9fa5f34372c`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:22.366215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:22.425662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1116/2241] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:22.441640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1126/2242] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:22.453402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:22.454704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:22.511890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d267d-24d9-4c16-a937-7568b7490faf None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:22.515660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:22.516335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:22.516621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:22.517190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:22.558429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:22.579605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:22.581107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:22.591015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1117/2243] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:24.102518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:24.104073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:24.117191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac55ac-f301-459e-9c75-f8415b1e2970 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:24.120825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:24.121503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:24.121788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:24.122070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:24.181327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:24.182778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:24.195541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1127/2244] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:24.778317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:24.789564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4fd52-c36b-4bb6-9be0-5eab05dd8749 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:24.795747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:24.796444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:24.796725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:24.797010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:24.811015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:24.812667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:24.823807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1118/2245] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:28.346083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:28.350493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:28.436226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:28.454408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb6813-ab03-41e5-b7de-7ede084e639f service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:28.461298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:28.462156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:28.462493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:28.462794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:28.465203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:28.560681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:28.562242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25440a0a-c076-4a3d-80e8-3408b9844a78 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:28.572302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1128/2246] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:28.973026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:29.053687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30930887-5d40-4ac5-8326-676d2bfa4d49 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:29.059033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:29.059905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:29.060405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:29.061548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] PATH_INFO: `/v3/users/0d7ed29ab94849a7a998d2cd957ae74f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:29.063598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7ed29ab94849a7a998d2cd957ae74f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:29.065994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:29.146555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7ed29ab94849a7a998d2cd957ae74f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:29.147559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7ed29ab94849a7a998d2cd957ae74f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:29.149890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7ed29ab94849a7a998d2cd957ae74f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:29.163230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7ed29ab94849a7a998d2cd957ae74f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:29.213753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7ed29ab94849a7a998d2cd957ae74f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:29.233190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1119/2247] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:28 2022] DELETE /identity/v3/users/0d7ed29ab94849a7a998d2cd957ae74f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:29.237518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:29.238093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:29.238591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:29.270747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:29.272012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:29.391103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1129/2248] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:29.437582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:29.439385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:29.512805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:29.527769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94dd5e7-8421-4f57-b106-ac0697d9c9c0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:29.533011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:29.533912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:29.534598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:29.534980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:29.548714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:29.550847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:29.562179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1120/2249] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:10:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:10:30.009194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:30.011108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:30.027862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519c10d-8e3a-465a-9114-31953ab1fa29 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:30.035206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:30.036468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:30.036825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:30.037233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] PATH_INFO: `/v3/projects/2e353963318c43b698d5d45c01a869f5` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:30.039689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e353963318c43b698d5d45c01a869f5)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:30.041945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:30.101995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e353963318c43b698d5d45c01a869f5', 'request_id': 'req-c99be917-1343-4fb4-a734-d6e4dafd9b2c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.113974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e353963318c43b698d5d45c01a869f5', 'request_id': 'req-c99be917-1343-4fb4-a734-d6e4dafd9b2c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.123091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e353963318c43b698d5d45c01a869f5', 'request_id': 'req-c99be917-1343-4fb4-a734-d6e4dafd9b2c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.137595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] The token cache is being invalidate because project 2e353963318c43b698d5d45c01a869f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:30.138143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.141010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.143787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1130/2250] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:30 2022] DELETE /identity/v3/projects/2e353963318c43b698d5d45c01a869f5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:30.148010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:30.160308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774565-7e10-4fe1-ae98-2b0f98c92587 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:30.237392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:30.238305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:30.238583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:30.238859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] PATH_INFO: `/v3/users/b76fb034ea324fb1b98a5b1e60c76c50` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:30.240085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76fb034ea324fb1b98a5b1e60c76c50)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:30.242976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:30.304480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76fb034ea324fb1b98a5b1e60c76c50'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.305519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76fb034ea324fb1b98a5b1e60c76c50'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.306184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76fb034ea324fb1b98a5b1e60c76c50'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.322534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76fb034ea324fb1b98a5b1e60c76c50'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.339046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76fb034ea324fb1b98a5b1e60c76c50'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.353259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1121/2251] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:30 2022] DELETE /identity/v3/users/b76fb034ea324fb1b98a5b1e60c76c50 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:10:30.774083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:30.776308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:30.792251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99be917-1343-4fb4-a734-d6e4dafd9b2c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:30.807954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:30.866247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:30.867371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:30.867808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:30.868297 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] PATH_INFO: `/v3/projects/4df990044bcc4efe801a6741c586e5b1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:30.869982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df990044bcc4efe801a6741c586e5b1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:30.871833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:30.918644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df990044bcc4efe801a6741c586e5b1', 'request_id': 'req-813be19b-c6bc-4b1c-8a48-625d7f2657d1'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.926876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df990044bcc4efe801a6741c586e5b1', 'request_id': 'req-813be19b-c6bc-4b1c-8a48-625d7f2657d1'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.932885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df990044bcc4efe801a6741c586e5b1', 'request_id': 'req-813be19b-c6bc-4b1c-8a48-625d7f2657d1'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.945210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] The token cache is being invalidate because project 4df990044bcc4efe801a6741c586e5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:30.945625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.946287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:30.947929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1131/2252] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:30 2022] DELETE /identity/v3/projects/4df990044bcc4efe801a6741c586e5b1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:30.951840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:30.953258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:30.965850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130b8d0-6dac-43b2-a007-cd268a6ad9b3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:31.023253 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:31.024571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:31.024862 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:31.025150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] PATH_INFO: `/v3/users/dfee7544125e456e810db9fa5f34372c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:31.026359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfee7544125e456e810db9fa5f34372c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:31.028346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:31.035025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:31.077916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfee7544125e456e810db9fa5f34372c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.078723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfee7544125e456e810db9fa5f34372c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.079426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfee7544125e456e810db9fa5f34372c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.087065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfee7544125e456e810db9fa5f34372c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.106152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d4e88367-d389-4f34-ba8b-eba31778a157 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfee7544125e456e810db9fa5f34372c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.129885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1122/2253] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:10:30 2022] DELETE /identity/v3/users/dfee7544125e456e810db9fa5f34372c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:31.436051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:31.437795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:31.452252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be19b-c6bc-4b1c-8a48-625d7f2657d1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:31.515436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:31.516332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:31.516625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:31.517163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] PATH_INFO: `/v3/projects/2780f57bad34411dbaa9031073d0f4cf` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:31.518349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2780f57bad34411dbaa9031073d0f4cf)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:31.520414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:31.634450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2780f57bad34411dbaa9031073d0f4cf', 'request_id': 'req-712c8984-42de-4f5a-8c1c-f769271b7950'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.638416 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:31.646811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2780f57bad34411dbaa9031073d0f4cf', 'request_id': 'req-712c8984-42de-4f5a-8c1c-f769271b7950'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.656933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2780f57bad34411dbaa9031073d0f4cf', 'request_id': 'req-712c8984-42de-4f5a-8c1c-f769271b7950'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.673910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] The token cache is being invalidate because project 2780f57bad34411dbaa9031073d0f4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:10:31.674216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.674956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:10:31.676907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1132/2254] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:10:31 2022] DELETE /identity/v3/projects/2780f57bad34411dbaa9031073d0f4cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:10:38.810991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683aa0c-1213-45eb-ac20-61da25bc6afd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:38.811502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683aa0c-1213-45eb-ac20-61da25bc6afd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:38.811609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6683aa0c-1213-45eb-ac20-61da25bc6afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:38.862551 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6683aa0c-1213-45eb-ac20-61da25bc6afd None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:38.863801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6683aa0c-1213-45eb-ac20-61da25bc6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:38.935363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1123/2255] 10.222.0.23 () {58 vars in 967 bytes} [Mon Jun 20 07:10:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:38.947580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:10:38.949319 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:39.040211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c8984-42de-4f5a-8c1c-f769271b7950 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:10:39.045955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:10:39.046607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:39.046892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:39.047171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:39.056056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:10:39.057671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:10:39.065923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1133/2256] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:10:39.697627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:10:39.697925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:10:39.698209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:10:39.716626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] MFA Rules not processed for user `f66183ca9d884f43befaace02099c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:10:39.717818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:10:39.773626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1124/2257] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:10:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:10:49.062475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=47 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:10:49.766682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:11:53.883450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:11:53.885010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:11:53.941648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298fe2a-28d7-4489-bb80-9895332e3221 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:11:53.945267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:11:53.945903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:11:53.946181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:11:53.946456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:11:53.948209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:11:54.019631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:11:54.021099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:11:54.031977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1134/2258] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:11:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:12:03.885491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:03.887485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:03.945166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ea57e-0c2b-42e7-80d3-56bb21a61b48 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:03.949203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:03.949835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:03.950115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:03.950394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:03.952176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:03.962589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:03.964526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be8d2b-04c8-4ba8-99d4-905466ca8937 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:03.973380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1125/2259] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:04.027356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=54 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:12:04.367022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:04.368454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:04.382453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227bbbe-7992-4ae0-883a-808de5a81026 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:04.388714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:04.389349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:04.389637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:04.389922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:04.391593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:04.449890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:04.451480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:04.460780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1135/2260] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:04.656440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:04.656738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:04.657025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:04.674679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] MFA Rules not processed for user `af0dee4e1dfb4f879dfe41622a9e23b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:12:04.675870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:04.731517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1126/2261] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:12:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:12:04.784635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:04.794528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30037ca8-7570-4968-800e-592cc68d8160 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:04.798607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:04.799356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:04.799647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:04.799922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:04.801783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:04.816015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:04.817469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:04.828476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1136/2262] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:04.875684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:04.887743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568dbdc-3832-4a1a-9ca3-2b0bcd4de4da None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:04.891331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:04.892270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:04.892559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:04.892836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:04.894643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:04.957822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:04.959481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:04.970535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1127/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:08.564267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:08.575835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ec6033-4586-4bed-9ce8-8ca18e2892ed service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:08.581289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:08.581921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:08.582208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:08.582487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:08.584286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:08.595039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:08.596640 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:08.605651 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1137/2264] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:14.965435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=79 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:12:18.601396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=40 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:12:38.738697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:12:38.761980 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afac11-c7cc-46ae-a530-ee9d2458bc2f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:12:38.769650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:12:38.770635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:12:38.771008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:12:38.771357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:12:38.774398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:12:38.800409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:12:38.802173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:12:38.815343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1128/2265] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:12:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:12:48.809541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:13:02.253384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:13:02.262781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aef5dfd-0129-4334-af0a-cfdcf7fd937a service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:13:02.266402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:13:02.267314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:13:02.267574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:13:02.267854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:13:02.269648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:13:02.279779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:13:02.281238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:13:02.289412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1138/2266] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:13:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:13:12.285677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:13:44.314538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:13:44.315958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:13:44.327392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab965aa-255d-4d6e-939c-31d8bf932039 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:13:44.330867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:13:44.331490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:13:44.331770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:13:44.332044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:13:44.333723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:13:44.390985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:13:44.392706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:13:44.402485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1129/2267] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:13:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:13:54.397381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=28 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:14:15.021667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:14:15.034437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf6ccb-2872-4903-9e8b-14d1b0769c0b service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:14:15.039775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:14:15.040507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:14:15.041303 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:14:15.041579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:14:15.043334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:14:15.099636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:14:15.101264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:14:15.110695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1139/2268] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:14:25.106473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=28 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:15:08.526757 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:15:08.528166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:15:08.598023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c8d8a-c3f2-4be1-8932-e36148443e62 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:15:08.603112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:15:08.603762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:15:08.604028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:15:08.604331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:15:08.606168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:15:08.621236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:15:08.622836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:15:08.632797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1130/2269] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:15:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:15:18.628532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=32 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:17:44.961572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:44.971357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c01ab-ca8f-4260-b09e-46d76ee1d88a service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:44.974919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:44.975582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:44.975864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:44.976194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:44.985510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:44.986931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.013430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1140/2270] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:17:45.069136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:45.081838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c77880-4d81-4ab7-9b15-e0cd5488cbea service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:45.085427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:45.086064 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:45.086349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:45.086627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:45.097539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:45.098960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.110316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1131/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:17:45.153254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:45.160946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b5d77-7915-4ebb-916a-f2775a2a6992 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:45.164592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:45.165868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:45.166163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:45.166443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:45.175174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:45.176631 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.203467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1141/2272] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:17:45.250529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:45.251968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:17:45.253098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:45.261787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5834ef-4690-4808-9652-63897756c1c1 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:45.265678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:45.266316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:45.266601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:45.266707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1796df-67ed-47d2-9379-63e80a2880dd service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:45.266886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:45.272313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:45.272884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:45.273172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:45.273889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:45.276316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:45.277735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.282693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:45.283738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.305818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1142/2273] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:17:45.314835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1132/2274] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:17:45.371626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:17:45.382439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc23ec0-1b42-44c5-9cba-9db96952aaa2 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:17:45.386266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:17:45.386979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:17:45.387350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:17:45.387712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:17:45.398580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:17:45.400136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:17:45.411450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1143/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:17:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:17:55.311851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=19 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:17:55.406247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=45 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:18:05.257873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:18:05.267996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c8bdf-6f0f-48bd-861e-a4e90d52eee6 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:18:05.271640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:18:05.272302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:18:05.272589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:18:05.272869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:18:05.279998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:18:05.281453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:18:05.289657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1133/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:18:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:18:15.285812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:19:45.618922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:19:45.624011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:19:45.637643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570c1fc7-da96-48c5-95c5-c095b445b651 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:19:45.642401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:19:45.643044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:19:45.643327 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:19:45.643604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:19:45.659571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:19:45.661022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:19:45.669054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6061-7094-4d83-85f2-f73cc7c1dd48 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:19:45.675248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:19:45.675932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:19:45.676256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:19:45.676559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:19:45.685498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:19:45.687088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:19:45.688192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1144/2277] 10.222.0.23 () {60 vars in 1396 bytes} [Mon Jun 20 07:19:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3248 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:19:45.725362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1134/2278] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:19:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:19:45.978324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:19:45.991180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9d4fb-0d71-4bcb-8115-e203fa1eeaa6 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:19:45.996572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:19:45.997202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:19:45.997489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:19:45.997769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:19:46.006615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:19:46.008215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1c143-4769-4b43-89d5-9726b7545eab service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:19:46.018824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1145/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:19:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:19:55.722603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=27 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:19:56.014462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=27 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:20:12.501610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:20:12.532036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd5d02-9f5e-46f6-b813-418b07dbc0f3 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:20:12.537976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:20:12.538669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:20:12.538954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:20:12.539232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:20:12.547871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:20:12.549481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:20:12.559118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1135/2280] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:20:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:20:22.555013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=13 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:22:41.913893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:41.914391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:41.914486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:41.960547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] MFA Rules not processed for user `f66183ca9d884f43befaace02099c7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:22:41.961661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:42.015693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1146/2281] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:22:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:22:42.319095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:42.320686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:42.396435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d0ec9-9d5f-4fec-a9c6-89bef4b5f5bf service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:42.400010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:42.400668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:42.400954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:42.401233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:42.402945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:42.412693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:42.414108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:42.421905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1136/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:42.741064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:42.753644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c68f39-019e-4c9b-a62e-33b3ca58c6ac None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:42.759575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:42.760259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:42.760546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:42.761078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:42.763066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:42.792252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:42.793828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:42.804081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1147/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:44.905284 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:44.937122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:44.938732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.023845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea509ce-73af-433a-b84e-409152de3124 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:45.032829 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:45.033493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:45.033914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:45.034726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:45.038157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.052033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9a6bf9-8da3-44e7-975c-cabcd39c6d5e service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:45.055450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:45.057140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:45.060389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:45.061783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:45.062513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:45.062995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:45.066695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.081733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1137/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:45.153882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:22:45.160545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:45.173693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad54a-e201-4ca7-8952-cc8c7a0f4086 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:45.179229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:45.180434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:45.180941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:45.181434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:45.185041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.187169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:45.189009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:45.199696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1148/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:22:45.200643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:45.202156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.240617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e3004-ff7e-457d-97ff-cc852501e362 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:45.248469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:45.248790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:45.250020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:45.250470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:45.250605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:45.251019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:45.254145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.269488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1138/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:45.362351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:45.368566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:45.379616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1149/2287] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:45.439814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:45.451450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86809e20-9499-4b31-96f1-181d80bd5f7e service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:45.457215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304af35-e510-4323-b752-603680748f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:45.457853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304af35-e510-4323-b752-603680748f11 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:45.458143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304af35-e510-4323-b752-603680748f11 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:45.459082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9304af35-e510-4323-b752-603680748f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:45.461009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9304af35-e510-4323-b752-603680748f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:45.487674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:22:45.538625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304af35-e510-4323-b752-603680748f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:45.540271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9304af35-e510-4323-b752-603680748f11 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:45.549455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1139/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:46.605359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:46.615224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5802e-ed3e-4ec1-820d-a94aae8b886e service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:46.620517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:46.621166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:46.621447 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:46.621722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:46.624237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:46.637716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:46.639135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:46.651956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1150/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:48.939816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304af35-e510-4323-b752-603680748f11 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:22:48.953027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9304af35-e510-4323-b752-603680748f11 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:22:48.958949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:22:48.959589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:22:48.959869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:22:48.960176 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:22:48.961951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:22:48.974430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:22:48.976006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:22:48.986775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1140/2290] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:22:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:22:56.645307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=54 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:22:58.982072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=26 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:23:11.222265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:23:11.230918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79fe5-41a5-4132-900d-9beffdc32d19 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:23:11.235014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:23:11.235692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:23:11.235977 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:23:11.236288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:23:11.238230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:23:11.248029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:23:11.249538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:23:11.258723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1151/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:23:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:23:21.253878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:24:44.219648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:24:44.221430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:24:44.232811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2976960-9655-4a27-bc98-d8b292bbc9ad service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:24:44.236494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:24:44.237132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:24:44.237421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:24:44.238201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:24:44.240181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:24:44.310718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:24:44.312186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:24:44.321189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1141/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:24:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:24:45.126758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:24:45.136241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7647d90-6040-4a45-8445-f9138c782a55 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:24:45.139813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:24:45.140479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:24:45.140764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:24:45.141041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:24:45.142898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:24:45.212181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:24:45.213603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:24:45.223100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1152/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:24:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:24:54.317037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=33 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:24:55.218393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=33 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:25:19.551499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:25:19.553106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:25:19.620590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7163142-b5d9-4515-b47c-9dd2ace45cc2 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:25:19.625646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:25:19.626276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:25:19.626558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:25:19.626832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:25:19.628688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:25:19.639949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:25:19.641549 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:25:19.651626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1142/2294] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:25:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:25:29.647513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=32 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:27:44.925125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:27:44.934592 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4363ebef-d72b-4237-93e2-2d9e10d9f103 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:27:44.938217 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:27:44.938928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:27:44.939215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:27:44.939498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:27:44.949891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:27:44.951346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:27:44.962598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1153/2295] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:27:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:27:44.969632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:27:44.979456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79205906-90f0-4e63-9446-c0531fa739d3 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:27:44.983276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:27:44.983906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:27:44.984230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:27:44.984513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:27:44.992988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:27:44.994409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:27:45.003381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1143/2296] 10.222.0.23 () {60 vars in 1396 bytes} [Mon Jun 20 07:27:44 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:27:45.565329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:27:45.575245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907caf5-325b-43f5-9d01-e0b6dbd5213f service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:27:45.578856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:27:45.579487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:27:45.579770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:27:45.580047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:27:45.588581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:27:45.590001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:27:45.624198 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1154/2297] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:27:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:27:45.670299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:27:45.680062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac80e9e-4cdb-489f-84e7-802435ced323 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:27:45.683659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:27:45.684322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:27:45.684610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:27:45.684892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:27:45.693738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:27:45.695168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:27:45.722167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1144/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:27:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:27:45.883953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:27:45.896469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f606cc9-c6a9-4ef0-a785-d8a8f080cd96 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:27:45.901949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:27:45.902582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:27:45.902866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:27:45.903145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:27:45.912246 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:27:45.913832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:27:45.944556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1155/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:27:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:27:55.719361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=19 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:27:55.942180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=32 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:28:19.210106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:19.221102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03473d9c-b7b0-4e10-9e7f-bae42cde5f4e service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:19.225445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:19.226086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:19.226372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:19.226657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:19.234421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:19.235840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:19.244378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1145/2300] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:28:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:28:29.240219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:28:40.646873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:40.656643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806880f-eb60-4052-964a-2a335d7bd5da service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:40.661014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:40.661672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:40.661956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:40.662562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:40.664494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:28:40.758967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:40.760549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:40.779300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1156/2301] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:28:41.277803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:41.287284 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf27d1a-d091-4877-9ea4-0a31bbf3e288 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:41.290961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:41.291600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:41.291885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:41.292202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:41.302613 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:41.304046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:41.315001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1146/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:28:44.253426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:44.264574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd590ce-1fe3-4223-ab16-6543e47892e5 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:44.269671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:44.270310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:44.270594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:44.270872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:44.272673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:28:44.284164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:44.285748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:44.309837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1157/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:28:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:28:47.446888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:47.448450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:28:47.463009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92661679-72ed-4571-aa46-8a90de52a725 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:47.467506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:47.468399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:47.468680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:47.468970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] PATH_INFO: `/v3/users/1404717904a14000a11f06c6c8291105` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:47.470141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1404717904a14000a11f06c6c8291105)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:47.472605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:47.519644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1404717904a14000a11f06c6c8291105'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.520449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1404717904a14000a11f06c6c8291105'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.521091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1404717904a14000a11f06c6c8291105'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.528261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1404717904a14000a11f06c6c8291105'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.541136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1404717904a14000a11f06c6c8291105'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.554502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1147/2304] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:28:47 2022] DELETE /identity/v3/users/1404717904a14000a11f06c6c8291105 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:28:47.772446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:47.784711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae99d28-6f05-4b5e-892b-0fbb3962443a service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:47.846992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:47.847862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:47.848179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:47.848464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] PATH_INFO: `/v3/projects/f5a98ee7cefc4c759c846f6cacb646a8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:47.849697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a98ee7cefc4c759c846f6cacb646a8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:47.851605 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:47.898749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a98ee7cefc4c759c846f6cacb646a8', 'request_id': 'req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.902150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:28:47.905883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a98ee7cefc4c759c846f6cacb646a8', 'request_id': 'req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.914981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a98ee7cefc4c759c846f6cacb646a8', 'request_id': 'req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.927219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] The token cache is being invalidate because project f5a98ee7cefc4c759c846f6cacb646a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:28:47.927512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.928242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:47.929823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1158/2305] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:28:47 2022] DELETE /identity/v3/projects/f5a98ee7cefc4c759c846f6cacb646a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:28:47.933815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:47.935215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:28:47.948365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32215e-0d5b-419c-88e1-dee3f83f83a7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:48.007621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:48.008526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:48.008807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:48.009077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] PATH_INFO: `/v3/users/66c18d8f9715494eb53e38ffd5c9531d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:48.025098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] RBAC: Authorizing `identity:delete_user(user_id=66c18d8f9715494eb53e38ffd5c9531d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:48.027699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:48.075791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66c18d8f9715494eb53e38ffd5c9531d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.076498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66c18d8f9715494eb53e38ffd5c9531d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.077111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66c18d8f9715494eb53e38ffd5c9531d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.079919 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:28:48.089289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66c18d8f9715494eb53e38ffd5c9531d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.106693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66c18d8f9715494eb53e38ffd5c9531d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.123533 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1148/2306] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:28:47 2022] DELETE /identity/v3/users/66c18d8f9715494eb53e38ffd5c9531d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:28:48.294915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:28:48.296380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:28:48.309696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d90af-4e0b-4fb2-8ddb-f870344a3d97 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:28:48.373013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:28:48.373876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:28:48.374163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:28:48.374432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] PATH_INFO: `/v3/projects/00f0938b938948fda228f78f1d06d071` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:28:48.388326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f0938b938948fda228f78f1d06d071)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:28:48.390252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:28:48.437717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f0938b938948fda228f78f1d06d071', 'request_id': 'req-9bcf28b8-804f-4f30-8158-ca1624d77ecc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.445533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f0938b938948fda228f78f1d06d071', 'request_id': 'req-9bcf28b8-804f-4f30-8158-ca1624d77ecc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.452765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f0938b938948fda228f78f1d06d071', 'request_id': 'req-9bcf28b8-804f-4f30-8158-ca1624d77ecc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.465040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] The token cache is being invalidate because project 00f0938b938948fda228f78f1d06d071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:28:48.465323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.465958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:28:48.467477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1159/2307] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:28:48 2022] DELETE /identity/v3/projects/00f0938b938948fda228f78f1d06d071 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:28:58.119485 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:28:58.461994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=61,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:29:31.419410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:29:31.421140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:29:31.562019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e35ec-91b1-41e7-aa1d-4ec73a44a70f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:29:31.645998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:29:31.646584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:29:31.647473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:29:31.647473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:29:31.649239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:29:31.662283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:29:31.663908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:29:31.677004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1149/2308] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:29:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:29:41.671173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=62 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:29:44.453808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:29:44.455301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:29:44.471441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf28b8-804f-4f30-8158-ca1624d77ecc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:29:44.476905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:29:44.477538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:29:44.477823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:29:44.478101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:29:44.479782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:29:44.536776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:29:44.538389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:29:44.547334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1160/2309] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:29:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:29:54.543222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=28 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:30:24.498504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:30:24.499893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:30:24.614019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a954aae-c2dc-4ef2-b07e-76df6093b77f service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:30:24.619591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:30:24.620260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:30:24.620545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:30:24.620827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:30:24.622576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:30:24.634672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:30:24.636260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65874214-2902-4b7f-a08c-97144ccdddaa service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:30:24.647886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1150/2310] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:30:34.642621 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=54 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:31:46.287442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:31:46.289321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:46.356464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ec63a-8f89-41b0-9fe3-c5ccd8936994 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:31:46.366340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:31:46.367121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:31:46.367599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:31:46.367969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:31:46.369989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:46.385038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:31:46.386661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:31:46.411368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1161/2311] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:31:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:31:47.797550 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:31:47.798089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:31:47.798089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:31:47.837086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] MFA Rules not processed for user `1c618f9ab8bd47a3a9913078f8ae895e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:31:47.838336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:47.908141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1151/2312] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:31:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:31:48.289037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:31:48.290458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:48.346503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a15117-a210-4dfa-a84e-7edd69a3b5cd service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:31:48.350735 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:31:48.351389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:31:48.351675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:31:48.351954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:31:48.353778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:48.364084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:31:48.365558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:31:48.374311 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1162/2313] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:31:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:31:48.726893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:31:48.739231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed0fe0-4ece-48c8-9293-6185dea2bb41 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:31:48.745963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:31:48.746687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:31:48.747037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:31:48.747526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:31:48.749392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:48.762041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:31:48.763707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:31:48.775568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1152/2314] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:31:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:31:55.411042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:31:55.426938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31958e-f40c-48b6-875b-9d50aaa228aa service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:31:55.433183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:31:55.433831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:31:55.434114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:31:55.434389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:31:55.436180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:31:55.448669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:31:55.450238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:31:55.461584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1163/2315] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:31:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:31:58.771846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=42 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:32:05.456450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=52 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:32:45.609345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:32:45.627461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d813d1-ac95-460f-a373-57520b3dbf54 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:32:45.631802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:32:45.632463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:32:45.632744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:32:45.633019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:32:45.634767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:32:45.676094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:32:45.677546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:32:45.697926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1153/2316] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:32:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:32:45.908970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:32:45.936023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53983d39-4e17-44b8-a539-ee3ec900868d service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:32:45.941410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:32:45.942035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:32:45.942314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:32:45.943215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:32:45.944973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:32:46.013459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:32:46.015059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:32:46.024225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1164/2317] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:32:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:32:55.695228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=26 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:32:56.019772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=40 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:33:25.200935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:33:25.209460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52696f-65ce-468e-8640-1ebea37d6608 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:33:25.213041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:33:25.213668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:33:25.214036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:33:25.214318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:33:25.216242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:33:25.238427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:33:25.240477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:33:25.248310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1154/2318] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:33:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:33:29.208808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:33:29.219958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80ccc9-76ae-41fa-8715-2c303c021c45 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:33:29.225897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:33:29.226538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:33:29.226822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:33:29.227104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:33:29.228870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:33:29.251865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:33:29.253728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:33:29.262771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1165/2319] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:33:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:33:35.244799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=11 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:33:39.258506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=11 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:33:42.100517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:33:42.108943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e8802-205e-440c-85f5-0b4817c98795 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:33:42.112583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:33:42.113204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:33:42.113492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:33:42.113774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:33:42.116059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:33:42.129401 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:33:42.131432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:33:42.142270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1155/2320] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:33:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:33:52.137493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:34:41.438872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:34:41.440401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:34:41.517353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f9c6c-8252-4a23-8164-46bdb4f14215 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:34:41.521561 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:34:41.522202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:34:41.522488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:34:41.522767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:34:41.524567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:34:41.592586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:34:41.594035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:34:41.603830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1166/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:34:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:34:41.645600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:34:41.654728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c162d-ee07-47ab-9cb4-4b5c7291970f service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:34:41.658874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:34:41.659593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:34:41.659883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:34:41.660197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:34:41.662573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:34:41.737594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:34:41.739080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:34:41.748384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1156/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:34:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:34:51.599366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=62 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:34:51.744059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=33 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:35:29.472687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:35:29.484382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcc091-7625-4034-909d-4377881fad33 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:35:29.489913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:35:29.490560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:35:29.490846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:35:29.491126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:35:29.499542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:35:29.501188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdcaaa-7b2d-4bd8-8eb0-2485936b2536 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:35:29.511925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1167/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:35:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:35:39.507938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:35:47.580721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:35:47.582593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:35:47.593815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b55c18-90e2-4bb3-b909-ee9bfecdae08 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:35:47.598320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:35:47.598974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:35:47.599263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:35:47.599548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:35:47.608345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:35:47.609784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:35:47.638561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1157/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:35:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:35:57.635412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=13 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:36:41.070965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:36:41.071459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:36:41.071647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:36:41.099182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] MFA Rules not processed for user `785140aa617e46608e8b3cf89aac5a44`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:36:41.100781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:36:41.107666 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1168/2325] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:36:41.410146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:36:41.419512 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a051451-2bd3-4308-af23-646e74c99e31 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:36:41.423035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:36:41.423666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:36:41.423948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:36:41.424262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:36:41.426014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:36:41.435154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:36:41.436607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:36:41.444538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1158/2326] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:36:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:36:44.591754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:36:44.605665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf2f97-3549-4da3-854e-d016e72a012d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:36:44.611367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:36:44.612096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:36:44.612488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:36:44.612864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:36:44.623242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:36:44.625048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:36:44.636937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1169/2327] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:36:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:36:51.441023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:36:54.632517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=12 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:37:46.007274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:37:46.015915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659e984-984d-459f-9cda-777266b7aebd service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:37:46.020018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:37:46.020677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:37:46.020976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:37:46.021238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:37:46.031258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:37:46.032768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:37:46.044084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1159/2328] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:37:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:37:56.039410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:38:28.489808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:38:28.500682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe1ad2-d3a9-4c61-8777-c45b8584ea2d service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:38:28.506975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:38:28.507596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:38:28.507878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:38:28.508184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:38:28.515967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:38:28.517564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:38:28.544567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1170/2329] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:38:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:38:30.242023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:38:30.250346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81f3fa-dfa4-44c9-bbdd-ef1a27842e60 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:38:30.254286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:38:30.254923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:38:30.255209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:38:30.255482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:38:30.262888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:38:30.264363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:38:30.273035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1160/2330] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:38:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:38:38.541933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=13 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:38:40.268756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:38:42.538310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:38:42.548014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41d413-d85f-4e45-a725-1becbec5f4e1 service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:38:42.552346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:38:42.552982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:38:42.553265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:38:42.553541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:38:42.562008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:38:42.563423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:38:42.572737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1171/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:38:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:38:43.487518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:38:43.495949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f69f0ff-2a2f-4b5d-976b-1534291ffda9 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:38:43.499708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:38:43.500378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:38:43.500926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:38:43.501209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:38:43.513711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:38:43.515145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:38:43.526923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1161/2332] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:38:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:38:52.568401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:38:53.521998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:39:41.933986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:39:42.003132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3419087-4ffd-4473-b6c6-715cba40923e service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:39:42.008780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:39:42.009627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:39:42.009944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:39:42.010227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:39:42.018682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:39:42.020358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:39:42.047248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1172/2333] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:39:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:39:52.044661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=41 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:40:34.479483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:40:34.481114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:40:34.582046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e8e4e-ca26-487c-9c54-97bb27e41bc0 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:40:34.587294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:40:34.587948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:40:34.588274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:40:34.588559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:40:34.590494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:40:34.602802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:40:34.604418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:40:34.614161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1162/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:40:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:40:42.728839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:40:42.738579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673730b-3aff-4626-b50c-648bc8e2e6ec service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:40:42.742415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:40:42.743093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:40:42.743378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:40:42.743657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:40:42.753745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:40:42.755165 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:40:42.782097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1173/2335] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:40:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:40:43.565105 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:40:43.566699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:40:43.582094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49753e2-a6d5-4c7d-81c6-48371495bcf8 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:40:43.587565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:40:43.588237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:40:43.588529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:40:43.588821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:40:43.598502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:40:43.600092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:40:43.610811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1163/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:40:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:40:52.779346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=13 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:40:53.606247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=51 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:44.515060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:44.524129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6dbf64-3748-4c37-847e-0c9d5779fde5 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:44.527929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:44.529345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:44.529766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:44.530187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:44.532893 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:44.609429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:44.610861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:44.619914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1174/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:46.077669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:46.079961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:46.101280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1fe1e-6cc0-4b04-b957-9e3eff93dba4 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:46.120252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:46.121855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:46.124653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:46.126754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:46.130663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:46.338411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:46.388760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a95f6b-47b1-4474-a8cc-94b420ce923e service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:46.399398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:46.400161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:46.400765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:46.400765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:46.404920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:46.422104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:46.425251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:46.430629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:46.433487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:46.459562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1175/2338] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:41:46.468037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1164/2339] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:47.006644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:47.033834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e65f46-7fab-4fad-8dd1-038c234c1d61 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:47.042588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:47.043254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:47.043540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:47.043824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:47.046312 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:47.059642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:47.061974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:47.073650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1176/2340] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:47.672045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:47.692420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa000ca-3a2a-4fdd-a3b1-cfdebc58fed0 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:47.699849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:47.701456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:47.701868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:47.702193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:47.704915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:47.720051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:47.721646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:47.734817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1165/2341] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:48.663992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:48.668670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:48.802654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0aa8ea-3a61-4a7f-b8b1-3a4d9a09adca service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:48.809750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:48.810923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:48.812673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:48.814719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:48.819682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:48.900780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:48.902799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:48.921033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1177/2342] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:50.327573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:50.329028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:50.357450 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804497a-c511-409f-8b2e-75c88aaed18b service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:50.367667 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:50.370814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:50.371368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:50.375090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:50.382208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:50.401166 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:50.403211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:50.435459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1166/2343] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:50.522237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:50.523981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:50.551907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7193b77-a700-4a78-adea-b406454ca378 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:50.557965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:50.558915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:50.559205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:50.559705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] PATH_INFO: `/v3/users/0dbf681122194384812c2eb96d1adfbc` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:50.562156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dbf681122194384812c2eb96d1adfbc)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:50.564315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:50.680833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:50.698958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dbf681122194384812c2eb96d1adfbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:50.711664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dbf681122194384812c2eb96d1adfbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:50.712826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dbf681122194384812c2eb96d1adfbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:50.727934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dbf681122194384812c2eb96d1adfbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:50.768778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c1f996c3-3220-487b-9398-e148a12bf19e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dbf681122194384812c2eb96d1adfbc'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:50.781145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1178/2344] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:41:50 2022] DELETE /identity/v3/users/0dbf681122194384812c2eb96d1adfbc => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:51.296793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:51.316433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d23e2-8a99-4cb9-afda-545a65d62178 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:51.453664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:51.455848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:51.457748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:51.459483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] PATH_INFO: `/v3/projects/1583bd5823544e54b32e1eaf308cd261` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:51.462378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1583bd5823544e54b32e1eaf308cd261)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:51.465243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:51.531946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:51.568591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1583bd5823544e54b32e1eaf308cd261', 'request_id': 'req-08d631fe-8028-4c45-8335-90ca043efe8b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:51.576835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1583bd5823544e54b32e1eaf308cd261', 'request_id': 'req-08d631fe-8028-4c45-8335-90ca043efe8b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:51.585798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1583bd5823544e54b32e1eaf308cd261', 'request_id': 'req-08d631fe-8028-4c45-8335-90ca043efe8b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:51.606190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] The token cache is being invalidate because project 1583bd5823544e54b32e1eaf308cd261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:41:51.606190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:51.607538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-08d631fe-8028-4c45-8335-90ca043efe8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:51.609546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1167/2345] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:41:51 2022] DELETE /identity/v3/projects/1583bd5823544e54b32e1eaf308cd261 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:51.615861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:51.616869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:51.617423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:51.757053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:51.758476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:51.830867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1179/2346] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:51.839432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:51.839986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:51.840487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:51.860784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:51.861998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:51.870271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1168/2347] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:51.875446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:51.886557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd562c9d-bad2-4929-a748-dc6c73e786aa None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:51.892345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:51.893020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:51.893306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:51.893585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:51.894321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:41:51.894815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:51.896636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:51.910443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1180/2348] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:41:51.915646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:51.929658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26640aa-a7d0-47eb-a9c6-1d06f846c2f6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:51.934542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:51.935946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:51.936456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:51.936908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:51.937825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:51.940280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.005709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1169/2349] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:51 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:52.011367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:52.023249 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010014ef-ec93-4cb9-ad91-244a3726f134 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:52.131652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:52.133018 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.133768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.134090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.136082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:52.138483 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.172713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1181/2350] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:52 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:52.176410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:52.188209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0aa11-feee-4dc6-8ba6-fbd066ed424d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:52.193557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:52.194240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.194527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.194878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.198731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:52.200516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.231390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:52.232653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1170/2351] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:52.237433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:52.246429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b2f4a-c552-492d-b0cf-3910bed0d7d6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:52.259041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:52.261426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.261737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.262334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.263434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:52.266376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f423539-2f47-4378-a292-c98197029c1f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.285678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1182/2352] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:52.287318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:52.324079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ade3-4861-491c-b313-c5f2430e061b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:52.333544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:52.334799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.335317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.335796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] PATH_INFO: `/v3/projects/0340346ca5a4424fa8a5374af75990da/users/eda8b1053f95458abfd8ea79227a0dcf/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.391872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0340346ca5a4424fa8a5374af75990da, user_id=eda8b1053f95458abfd8ea79227a0dcf, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:52.395631 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.440383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1171/2353] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:52 2022] PUT /identity/v3/projects/0340346ca5a4424fa8a5374af75990da/users/eda8b1053f95458abfd8ea79227a0dcf/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:52.442714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.443185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.443641 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.475222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:52.476972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:52.593477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1183/2354] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:41:52.709229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:52.712992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:52.892312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b224bf-c6c0-4cdc-82aa-19af7c415630 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:52.902300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:52.904996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:52.905169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:52.905417 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:52.941964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:52.945977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:52.985742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1172/2355] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:53.342387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:53.343945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:53.361871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedb2e2-0ee7-4639-b893-24aae36225a3 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:53.367479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:53.373011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:53.373766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:53.374584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] PATH_INFO: `/v3/users/785140aa617e46608e8b3cf89aac5a44` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:53.377439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=785140aa617e46608e8b3cf89aac5a44)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:53.381560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:53.502635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785140aa617e46608e8b3cf89aac5a44'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:53.512755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785140aa617e46608e8b3cf89aac5a44'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:53.514125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785140aa617e46608e8b3cf89aac5a44'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:53.531161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785140aa617e46608e8b3cf89aac5a44'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:53.567797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-bb730d6a-3cf0-453a-b584-f138c0eba81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785140aa617e46608e8b3cf89aac5a44'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:53.583905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1184/2356] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:41:53 2022] DELETE /identity/v3/users/785140aa617e46608e8b3cf89aac5a44 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:53.844750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:53.857109 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78c9da-38b9-4602-aa3b-4d7bb0fa5ec1 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:53.904404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:53.939966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:53.941337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:53.941860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:53.942243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] PATH_INFO: `/v3/projects/0e19a3f933284f088747a9d623e9ec24` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:53.944017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e19a3f933284f088747a9d623e9ec24)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:53.946738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.013595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e19a3f933284f088747a9d623e9ec24', 'request_id': 'req-632f340c-5a61-4cf7-90f0-a75f6e232cad'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.022301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e19a3f933284f088747a9d623e9ec24', 'request_id': 'req-632f340c-5a61-4cf7-90f0-a75f6e232cad'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.034703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e19a3f933284f088747a9d623e9ec24', 'request_id': 'req-632f340c-5a61-4cf7-90f0-a75f6e232cad'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.073239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] The token cache is being invalidate because project 0e19a3f933284f088747a9d623e9ec24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:41:54.073773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.074441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.076915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1173/2357] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:41:53 2022] DELETE /identity/v3/projects/0e19a3f933284f088747a9d623e9ec24 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:54.083900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3944f-d3fc-490e-aaf0-1116e83b7597 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.084255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3944f-d3fc-490e-aaf0-1116e83b7597 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.084565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3944f-d3fc-490e-aaf0-1116e83b7597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.133192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ede3944f-d3fc-490e-aaf0-1116e83b7597 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:54.136469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ede3944f-d3fc-490e-aaf0-1116e83b7597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:54.292166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1185/2358] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:54.292383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.292383 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:54.297676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.298082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.298471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.306978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:54.323274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:54.324806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:54.348625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1186/2359] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:54.350912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.368038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a1ec1-f210-49c2-8e68-47d28c2860df None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.377522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.379424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.379943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.380836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.383029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:41:54.383754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.404625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.409230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f340c-5a61-4cf7-90f0-a75f6e232cad admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.413890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.414545 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.414828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.415101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.422059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:54.441662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1187/2360] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:41:54.443718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.446066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.448869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.462065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691d1da-bf6a-451b-87cc-f1639afa077a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.467657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.468557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.469010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.469477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.470997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.473138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.482945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1174/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:41:54.523569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1188/2362] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:54 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:54.527539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.555857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b6b55-fb8d-4c67-bd9d-76860ec47705 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.631073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.632402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.633146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.633530 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.634635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.637134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.663202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:54.681802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1175/2363] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:54 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:54.684613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.698892 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b83eb-ad77-4617-abb1-b6c05ea8a134 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.705109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.706197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.706726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.707232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.708296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.711334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.727537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1189/2364] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:54.732162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.750367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.759183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc48093-4be4-4064-bde4-7e04f1865979 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.762563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:54.784317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.785007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.785290 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.785567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.787030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.788713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.798325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed38d0-a3b2-4f2c-9de9-631fedb59827 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.819471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1176/2365] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:54.820972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.821859 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.822147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.822512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] PATH_INFO: `/v3/users/f66183ca9d884f43befaace02099c7f2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.824945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66183ca9d884f43befaace02099c7f2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.830777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:54.831706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.850676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7783f08-bc97-4a11-a3c0-ab800cd37c3b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:54.857476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:54.858718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.859287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.860722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] PATH_INFO: `/v3/projects/c7ddb21614eb44f4baeafacd2e3baead/users/f735526cee0b4837862ad8e60277b7b3/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.889001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ddb21614eb44f4baeafacd2e3baead, user_id=f735526cee0b4837862ad8e60277b7b3, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:54.893282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09c9276-9c1a-4064-9e4c-e17696054a7f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:54.928629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1177/2366] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:54 2022] PUT /identity/v3/projects/c7ddb21614eb44f4baeafacd2e3baead/users/f735526cee0b4837862ad8e60277b7b3/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:54.929250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66183ca9d884f43befaace02099c7f2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.940373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:54.940735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:54.940970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:54.950065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66183ca9d884f43befaace02099c7f2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.951344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66183ca9d884f43befaace02099c7f2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:54.970902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66183ca9d884f43befaace02099c7f2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.000098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66183ca9d884f43befaace02099c7f2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.010852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1190/2367] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:41:54 2022] DELETE /identity/v3/users/f66183ca9d884f43befaace02099c7f2 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:55.013146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] MFA Rules not processed for user `f735526cee0b4837862ad8e60277b7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:55.014594 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.102349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1178/2368] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:55.115033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:55.116978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.248336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e92e7a8-9d37-4d23-85fc-ce86fd06f5fd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:55.252416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:55.254066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.258104 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:55.259105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:55.260959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:55.261698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:55.277705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:55.281674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:55.307409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1191/2369] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:55.393135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:55.394790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.419555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2d71-5036-4fc0-93f2-8b519079fc60 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:55.423025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:55.449623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:55.450041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:55.451086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:55.451547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:55.458237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.485387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:55.485522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:55.505243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1179/2370] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:55.506091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:55.510499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:55.588159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c72ab4-6f4e-4d3f-bdcf-179c00a1b30c service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:55.604543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:55.605983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:55.605983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:55.606252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] PATH_INFO: `/v3/users/1c618f9ab8bd47a3a9913078f8ae895e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:55.608175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c618f9ab8bd47a3a9913078f8ae895e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:55.609645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:55.615997 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:55.657059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73cd08-0599-4a35-b85a-76a203e28702 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:55.664685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:55.671641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:55.671972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:55.673891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] PATH_INFO: `/v3/projects/31415828d04241249f14fb3ef6a3aa0a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:55.677727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=31415828d04241249f14fb3ef6a3aa0a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:55.680783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:55.762304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c618f9ab8bd47a3a9913078f8ae895e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.775069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c618f9ab8bd47a3a9913078f8ae895e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.776915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c618f9ab8bd47a3a9913078f8ae895e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.789203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c618f9ab8bd47a3a9913078f8ae895e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.813306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31415828d04241249f14fb3ef6a3aa0a', 'request_id': 'req-bf26c25e-7a95-412f-a643-ea2d948693eb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.826759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31415828d04241249f14fb3ef6a3aa0a', 'request_id': 'req-bf26c25e-7a95-412f-a643-ea2d948693eb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.827304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c355b0e5-1c7b-4284-8708-cb50c3ec998c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c618f9ab8bd47a3a9913078f8ae895e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.846606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31415828d04241249f14fb3ef6a3aa0a', 'request_id': 'req-bf26c25e-7a95-412f-a643-ea2d948693eb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.849926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1192/2371] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:41:55 2022] DELETE /identity/v3/users/1c618f9ab8bd47a3a9913078f8ae895e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:55.880504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] The token cache is being invalidate because project 31415828d04241249f14fb3ef6a3aa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:41:55.880993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.884320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-bf26c25e-7a95-412f-a643-ea2d948693eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:55.889396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1180/2372] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:41:55 2022] DELETE /identity/v3/projects/31415828d04241249f14fb3ef6a3aa0a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:55.897870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:55.898118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:55.901870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:55.930963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:55.933290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:56.098285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1193/2373] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:56.106856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.107489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.107859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.137783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:56.140408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:56.161263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1181/2374] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:56.165383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.176458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33c154-ba40-4d14-98ee-e976df7e3c2c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.182590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.183706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.184280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.185213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.185965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:41:56.186270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.187914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.206277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1194/2375] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:41:56.207837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.219203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6aafb9-47ee-4967-b50b-083412376339 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.224996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.225432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.225718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.225994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.226689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.228956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.263974 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1182/2376] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:41:56.267954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.281999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528d296-5211-4ae7-90d2-c451d998a889 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.298105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:56.390483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.393725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:56.433085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.433799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.434090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.434374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.435670 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.437644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.485153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1195/2377] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:56.489379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.506461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fe791-d880-4ca7-88aa-bf7091e1a6df admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.512464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.513503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.513962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.514415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.520389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e980f-11df-4b8d-a5bd-53cf87023926 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.522692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.525323 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.525884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.526870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.527259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.527633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] PATH_INFO: `/v3/projects/530da3341ec44dddbab5b0042706ac7b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.541899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530da3341ec44dddbab5b0042706ac7b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.543129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1196/2378] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:56.545074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.549229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.565127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fceb3-6550-4c3a-ab9f-7189ad93d138 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.574843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.575927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.577067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.577544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.578507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.580674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.594733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1197/2379] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:56.598461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:56.598978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:56.621671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131c65c-f11a-4712-b2a4-8118e92ed2b0 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:56.628783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530da3341ec44dddbab5b0042706ac7b', 'request_id': 'req-3c6c0218-bb2c-4910-9415-79edb6be6b81'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:56.637808 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530da3341ec44dddbab5b0042706ac7b', 'request_id': 'req-3c6c0218-bb2c-4910-9415-79edb6be6b81'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:56.650144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530da3341ec44dddbab5b0042706ac7b', 'request_id': 'req-3c6c0218-bb2c-4910-9415-79edb6be6b81'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:56.675199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] The token cache is being invalidate because project 530da3341ec44dddbab5b0042706ac7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:41:56.675840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:56.676899 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3c6c0218-bb2c-4910-9415-79edb6be6b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:41:56.682588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1183/2380] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:41:56 2022] DELETE /identity/v3/projects/530da3341ec44dddbab5b0042706ac7b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:56.685296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be458-0f32-419c-857b-c45781d98fcd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.685798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be458-0f32-419c-857b-c45781d98fcd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.686327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562be458-0f32-419c-857b-c45781d98fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.712923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-562be458-0f32-419c-857b-c45781d98fcd None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:56.714528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-562be458-0f32-419c-857b-c45781d98fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:56.762488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:56.764575 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.765034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.765476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] PATH_INFO: `/v3/projects/00239053d31046c493ca68ea15431fa3/users/36ac1fd0310c4eb8889a82628b2580a7/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.792994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00239053d31046c493ca68ea15431fa3, user_id=36ac1fd0310c4eb8889a82628b2580a7, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:56.796518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122958ce-e5e6-43dd-b1e6-a4c1d7d2e0d8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:56.814779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1198/2381] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:56 2022] PUT /identity/v3/projects/00239053d31046c493ca68ea15431fa3/users/36ac1fd0310c4eb8889a82628b2580a7/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:56.826897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.827440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.827832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.848434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1184/2382] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:56.857330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:56.863153 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:56.863457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:56.888480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:56.889871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:56.905331 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] MFA Rules not processed for user `36ac1fd0310c4eb8889a82628b2580a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:56.907607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:57.003570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1185/2383] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:57.010218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.022695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:57.046193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd027b-30ec-4e36-a5e8-98c29c0bc26e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.049315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1199/2384] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:41:57.052275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.058903 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:57.065807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.066497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.068723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.069044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.069818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:41:57.070139 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.073045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.089883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd47a0ee-d597-4b94-83e5-dcf186133251 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.104275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1186/2385] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:41:57.106387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.106839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.107126 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.107409 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.108193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.108720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.110560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.130135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b5267f-9f28-43c3-b625-f920c12b6d8c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.144624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.146305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.146608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.146895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.147615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.150586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.188446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1200/2386] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:57 2022] POST /identity/v3/projects => generated 391 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:41:57.193021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.214071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1187/2387] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:57 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:57.217777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.319287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:57.341665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb63b99-9102-4af0-be72-71280d4d0350 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.385805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2260d-6f40-4dc7-99d3-83032568e3cf admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.404882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.405583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.405873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.406155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.406870 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.408921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.468513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.469842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.470395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.470847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.471728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.473948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.475766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1188/2388] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:57 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:57.476841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.501892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6115f26-9f5a-430e-b083-f5ed86a71098 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.511345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.512321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.518297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.519736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.520772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1201/2389] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:57 2022] POST /identity/v3/users => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:57.524822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.525114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.529227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.551976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17ff9d-94a5-4cb8-a5ce-dae8defcd749 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.557237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.558489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.558974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.559978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.563356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1189/2390] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:57.563957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.564385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.569003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.589801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2821b-5f8c-473f-a6c4-52d66599094b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.594724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1202/2391] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:57.599127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.600701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.604627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.604936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.606898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.608576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.615670 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.644284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca9e695-7336-4f45-b3cc-b230d16bfb6a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.686028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1190/2392] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:57.689252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.691353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.696144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.696144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.696144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] PATH_INFO: `/v3/projects/d047721f526e4940b6161807f0c0928b/users/a7588aa439dc43449979fbb771ab531d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.729317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16891cef-61d6-44c9-9488-ff219410fc78 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:57.738915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:57.741329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.741864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.743308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] PATH_INFO: `/v3/projects/0afa24eade4c44cc9810a535cc209739/users/e1da04de978248c38b6af3f057a78fb6/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.755758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d047721f526e4940b6161807f0c0928b, user_id=a7588aa439dc43449979fbb771ab531d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.758924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.784622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1203/2393] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:57 2022] PUT /identity/v3/projects/d047721f526e4940b6161807f0c0928b/users/a7588aa439dc43449979fbb771ab531d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:41:57.789375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:57.800432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0afa24eade4c44cc9810a535cc209739, user_id=e1da04de978248c38b6af3f057a78fb6, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:57.807926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db63b9e-036f-4d20-9690-91312000bd15 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:57.855151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1191/2394] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:57 2022] PUT /identity/v3/projects/0afa24eade4c44cc9810a535cc209739/users/e1da04de978248c38b6af3f057a78fb6/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:57.857423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:57.857857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:57.858242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:57.884944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:57.917807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] MFA Rules not processed for user `e1da04de978248c38b6af3f057a78fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:57.923102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:57.935460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d8ab0-a64f-41ef-9abf-ff1519393895 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.060008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1192/2395] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:58.064372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.066672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.067550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.067839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.068135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.068876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.070501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.090112 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be54f-fd0c-484f-afd3-9e43ef57873b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.101169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1204/2396] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:41:58.101927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.109494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.110529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.110529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.110761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.111437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.113387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.122350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9062640-39ae-4f85-80fe-83002da6bc07 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.127966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.128907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.129191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.129467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] PATH_INFO: `/v3/projects/d047721f526e4940b6161807f0c0928b/users/a7588aa439dc43449979fbb771ab531d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.131719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d047721f526e4940b6161807f0c0928b, user_id=a7588aa439dc43449979fbb771ab531d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.134457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7d142-d0a4-48b1-8b27-00720b216d28 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.166415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1205/2397] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:58 2022] PUT /identity/v3/projects/d047721f526e4940b6161807f0c0928b/users/a7588aa439dc43449979fbb771ab531d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:41:58.182491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.182845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.183700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.189108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1193/2398] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:41:58 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:58.190101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.229209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] MFA Rules not processed for user `a7588aa439dc43449979fbb771ab531d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:58.231412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:58.312804 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:58.346249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340f3e1-45b2-46e6-8751-d1bde45b1cd8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.355983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.356727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.357011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.357287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.357989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.360101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.408241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1206/2399] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:58.408700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1194/2400] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:41:58 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:41:58.415114 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.422679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.425801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:58.435602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54034a13-bcbf-4331-8ae6-f73fc8c69d66 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.455613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.457200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.458211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.458596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.461556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.463601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.498294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1207/2401] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:41:58.502659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.514209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8015a7-fc6f-4260-bc41-b0c20b94beb4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.519112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.520302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.520585 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.520862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] PATH_INFO: `/v3/projects/3bd3042725d547899219355da19da1e4/users/9a5fcdb34ba7418483bcbfb531024782/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.538834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b70704-2080-4db2-a405-0230085db140 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.543642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.544681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.545137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.545587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.549235 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd3042725d547899219355da19da1e4, user_id=9a5fcdb34ba7418483bcbfb531024782, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.552324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.563891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.568915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.581082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1208/2402] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:58 2022] PUT /identity/v3/projects/3bd3042725d547899219355da19da1e4/users/9a5fcdb34ba7418483bcbfb531024782/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:41:58.585535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.613327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1195/2403] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:41:58.631852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:58.699951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.702037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:58.726243 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589d4a2-67a5-4aa5-b5bc-62f4e35b198e service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.741740 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816a4c-2c2e-4521-aafe-dc306df28f82 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.756026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.759101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.759101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.759685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.761296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.767136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.794468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1209/2404] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:41:58.798501 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:58.828434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4eff2-1534-426c-89b6-5b8a0d9a3706 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:58.842048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.843175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.843632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.843765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] PATH_INFO: `/v3/projects/3bd3042725d547899219355da19da1e4/users/9a5fcdb34ba7418483bcbfb531024782/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.852374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd3042725d547899219355da19da1e4, user_id=9a5fcdb34ba7418483bcbfb531024782, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:58.854554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709a501-fb8a-4f2b-a1e7-3b0bb18a9f14 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:58.859082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:58.860067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.860236 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.860508 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.886024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1210/2405] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:41:58 2022] PUT /identity/v3/projects/3bd3042725d547899219355da19da1e4/users/9a5fcdb34ba7418483bcbfb531024782/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:41:58.899465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:58.899797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:58.900503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:58.936913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] MFA Rules not processed for user `9a5fcdb34ba7418483bcbfb531024782`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:41:58.938200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:41:59.034403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:59.035992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:59.060518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1196/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:41:59.112799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1211/2407] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:41:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:41:59.125927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:41:59.137708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb4234-e570-42e7-94a7-ea48f4916a7c service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:41:59.226172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:41:59.284898 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:41:59.285818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:41:59.286471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:41:59.286927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:41:59.305818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:41:59.307907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5869146-1058-4927-b2dd-17445d026dbb service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:41:59.344432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1197/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:41:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:00.004263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.142069 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe851f-eb4e-4a13-a5df-8149ae805f94 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.150181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.151230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.151521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.151802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] PATH_INFO: `/v3/users/e1da04de978248c38b6af3f057a78fb6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.154034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1da04de978248c38b6af3f057a78fb6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:00.164277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:00.221888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:00.292285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1da04de978248c38b6af3f057a78fb6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:00.314152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1da04de978248c38b6af3f057a78fb6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:00.315487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1da04de978248c38b6af3f057a78fb6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:00.324834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1da04de978248c38b6af3f057a78fb6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:00.346822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-14720e8c-75a2-4a04-870a-fc09a90bee74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1da04de978248c38b6af3f057a78fb6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:00.361291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1212/2409] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:00 2022] DELETE /identity/v3/users/e1da04de978248c38b6af3f057a78fb6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:00.365503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.366322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.366604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.382997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.383343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.383570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.411822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:00.414890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] MFA Rules not processed for user `eda8b1053f95458abfd8ea79227a0dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:00.416383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:00.429305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:00.523376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1213/2410] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:00.525289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.527389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:00.568527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1198/2411] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:00.584959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.588451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:00.623266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6fd4-daae-4bbe-9a2a-03664c27567c None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.628721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.629417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.629856 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.630414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.631146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:00.633921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:00.678318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1214/2412] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:00 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:00.697680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.704033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd21a1-258e-465f-a74b-a8287f771c40 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.749267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:00.813695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c2d6e-237f-4225-9034-0bf9c010aa0e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.819699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.820583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.820871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.821246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.822320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.822994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.823284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.823942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.824295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:00.826429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:00.832016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:00.834010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:00.851177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1199/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:00.869507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1215/2414] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:00 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:00.874967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.890798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c4bc7-2a36-4bf4-a9b8-03669d0577db service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.898299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.899194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.899588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.900046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.901749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:00.905969 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:00.924577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1200/2415] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:00.926840 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:00.955855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87a396-290e-4bd4-aa21-cb402af130d7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:00.972607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:00.974385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:00.974676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:00.974973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] PATH_INFO: `/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8/users/a091e587a9c547cca66b526425afc2c8/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:00.998843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d82c5d67dc34d7ba1c7b6c744e0d7a8, user_id=a091e587a9c547cca66b526425afc2c8, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.003998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.022742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1216/2416] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:00 2022] PUT /identity/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8/users/a091e587a9c547cca66b526425afc2c8/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:01.028753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.041388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484287a0-1776-43b5-9102-1968ddd09d8c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.068614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:01.120854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.121591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.122086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.122251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.123063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.125994 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.141632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1201/2417] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:01.145786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.162648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.165561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:01.182905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a76ac0-7677-4d26-b1ea-d84ef4b8f3d3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.222196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.223222 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.223511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.223789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] PATH_INFO: `/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8/users/a091e587a9c547cca66b526425afc2c8/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.226955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb8389-8e8a-4330-91c6-a9fa7ae3776d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.229054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d82c5d67dc34d7ba1c7b6c744e0d7a8, user_id=a091e587a9c547cca66b526425afc2c8, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.232978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.240996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.241956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.242234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.242626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] PATH_INFO: `/v3/users/af0dee4e1dfb4f879dfe41622a9e23b1` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.247940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0dee4e1dfb4f879dfe41622a9e23b1)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.250676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.268895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1217/2418] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:01 2022] PUT /identity/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8/users/a091e587a9c547cca66b526425afc2c8/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:01.320098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.321821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:01.364163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0dee4e1dfb4f879dfe41622a9e23b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.365080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0dee4e1dfb4f879dfe41622a9e23b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.366085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0dee4e1dfb4f879dfe41622a9e23b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.383058 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0dee4e1dfb4f879dfe41622a9e23b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.399357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0dee4e1dfb4f879dfe41622a9e23b1'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.419031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1202/2419] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:01 2022] DELETE /identity/v3/users/af0dee4e1dfb4f879dfe41622a9e23b1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:01.449118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54a037-60cf-46fa-b92c-f32326178449 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.538970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.541186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.542029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.542320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] PATH_INFO: `/v3/projects/0afa24eade4c44cc9810a535cc209739` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.545791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0afa24eade4c44cc9810a535cc209739)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.547858 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.587050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:01.626818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0afa24eade4c44cc9810a535cc209739', 'request_id': 'req-7a6ece5b-889a-47e1-9e24-88cc16d5f931'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.650545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0afa24eade4c44cc9810a535cc209739', 'request_id': 'req-7a6ece5b-889a-47e1-9e24-88cc16d5f931'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.663738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.665680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0afa24eade4c44cc9810a535cc209739', 'request_id': 'req-7a6ece5b-889a-47e1-9e24-88cc16d5f931'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.671848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:01.679038 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] The token cache is being invalidate because project 0afa24eade4c44cc9810a535cc209739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:01.679343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.680029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.684608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1218/2420] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:01 2022] DELETE /identity/v3/projects/0afa24eade4c44cc9810a535cc209739 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:01.686043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:01.688374 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:01.727971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2355c-10a2-4a06-9750-bb1940d39451 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.797434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ece5b-889a-47e1-9e24-88cc16d5f931 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:01.803842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.805077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.805568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.806048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] PATH_INFO: `/v3/users/9a5fcdb34ba7418483bcbfb531024782` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.807751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5fcdb34ba7418483bcbfb531024782)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.810500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.885165 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:01.888579 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:01.888909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:01.889223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] PATH_INFO: `/v3/projects/b07b7ec64ab448338374bb16c29c7b1d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:01.890644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b07b7ec64ab448338374bb16c29c7b1d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:01.893087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:01.924443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5fcdb34ba7418483bcbfb531024782'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.964316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5fcdb34ba7418483bcbfb531024782'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.965583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5fcdb34ba7418483bcbfb531024782'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.965832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:01.978601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5fcdb34ba7418483bcbfb531024782'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:01.992634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b07b7ec64ab448338374bb16c29c7b1d', 'request_id': 'req-38e805a5-cee4-4b78-95ad-de7487373108'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.006075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b07b7ec64ab448338374bb16c29c7b1d', 'request_id': 'req-38e805a5-cee4-4b78-95ad-de7487373108'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.018951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2d0b2a48-cf0e-4126-b46e-dd90b6f310a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5fcdb34ba7418483bcbfb531024782'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.021405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b07b7ec64ab448338374bb16c29c7b1d', 'request_id': 'req-38e805a5-cee4-4b78-95ad-de7487373108'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.033088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1219/2421] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:01 2022] DELETE /identity/v3/users/9a5fcdb34ba7418483bcbfb531024782 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:02.044235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] The token cache is being invalidate because project b07b7ec64ab448338374bb16c29c7b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:02.044975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.046309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-38e805a5-cee4-4b78-95ad-de7487373108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:02.048643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1203/2422] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:01 2022] DELETE /identity/v3/projects/b07b7ec64ab448338374bb16c29c7b1d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:02.055395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.055988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.056562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.083442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:02.085184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:02.116706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:02.211422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1220/2423] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:02.214787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.216988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.217942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.245483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:02.250371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:02.267891 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1204/2424] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:02.272239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.283520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c284dc-ff9f-4006-ab28-766e584c14cc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.293128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.295042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.295618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.296576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.297940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:02.298636 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.300786 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.391434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.404685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1221/2425] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:02.414256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c907a-026e-4ce3-a954-fae92405c7c2 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.438475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.439156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.439444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.439843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.440577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.444177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.510896 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1205/2426] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:02 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:02.514793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.531315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d16139-e0e7-4e7f-b364-848962f1fd84 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.663178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.664589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.665207 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.665888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.666810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.670646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.710027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.724369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed656c9-5812-43b5-8923-e22129b80832 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.725019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1222/2427] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:02 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:02.735125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.736263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.736750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.737040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.737805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.739681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.759749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1206/2428] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:02.764065 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.780231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cb9bc-7d42-4e9e-b057-e4c6fd87a342 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.803031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.804299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.805776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.806256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.807180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.809547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4986b0e-4549-40f5-b241-15edf44d2869 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.830979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:02.835703 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1223/2429] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:02.852663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7458ed-2d56-4372-8ed7-cd8eae73bf1c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:02.869269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:02.870830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.871394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.879851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] PATH_INFO: `/v3/projects/758a80f1cf56414582066891009ab9dd/users/b77f0d7681024db28eef64a86dd2aea2/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:02.917056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758a80f1cf56414582066891009ab9dd, user_id=b77f0d7681024db28eef64a86dd2aea2, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:02.922357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:02.969233 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1207/2430] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:02 2022] PUT /identity/v3/projects/758a80f1cf56414582066891009ab9dd/users/b77f0d7681024db28eef64a86dd2aea2/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:02.972638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:02.972932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:02.974066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.017881 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] MFA Rules not processed for user `b77f0d7681024db28eef64a86dd2aea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:03.019111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:03.052651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.054525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:03.154963 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1224/2431] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:03.159779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.235762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:03.255344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93acd9dc-a6a9-4339-83c6-dc5d29a614f1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.277605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.279041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.279325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.279578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] PATH_INFO: `/v3/projects/3bd3042725d547899219355da19da1e4` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.281133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd3042725d547899219355da19da1e4)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.283515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.318367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf2d6-066d-430e-86ca-4c6822d4a6cb None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.327023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.327706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.328026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.328379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.329513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.332665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.357182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd3042725d547899219355da19da1e4', 'request_id': 'req-26eba7d4-af1d-451f-9219-c750c2560973'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:03.366199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd3042725d547899219355da19da1e4', 'request_id': 'req-26eba7d4-af1d-451f-9219-c750c2560973'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:03.369454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:03.393413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd3042725d547899219355da19da1e4', 'request_id': 'req-26eba7d4-af1d-451f-9219-c750c2560973'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:03.407748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1225/2432] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/projects => generated 363 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:03.415604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.439747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] The token cache is being invalidate because project 3bd3042725d547899219355da19da1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:03.441098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:03.442203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-26eba7d4-af1d-451f-9219-c750c2560973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:03.445619 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1208/2433] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:03 2022] DELETE /identity/v3/projects/3bd3042725d547899219355da19da1e4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:03.453097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ca58-526d-4b07-bd59-280482347297 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.453642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ca58-526d-4b07-bd59-280482347297 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.454106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832ca58-526d-4b07-bd59-280482347297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.491439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-e832ca58-526d-4b07-bd59-280482347297 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:03.492689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e832ca58-526d-4b07-bd59-280482347297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:03.535363 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5458887-de14-4331-9465-32ab0e40f6d8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.549000 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.550079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.551139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.551678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.552999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.556330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.589065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1209/2434] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:03.603641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.603943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.604421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.606478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1226/2435] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:03.608064 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.609793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:03.625756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9696b-d2bf-40be-9129-8f323c65ed0a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.634207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:03.635611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:03.644525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.645473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.645770 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.646086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.647117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.648874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.650328 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1210/2436] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:03.655368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.668089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1227/2437] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:03.668322 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe9127-093e-45d3-ad4b-976637bb52df None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.669183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.675469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.676404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.676756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.677101 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.680793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:03.681229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.683164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.716314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5726b3-69bf-40e5-b2ef-53a6fd96cf07 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.723504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1211/2438] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:42:03.723504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.730094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.730094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.731125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.731125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] PATH_INFO: `/v3/projects/18ccaae6b5064751928d74cf9b45b9b7/users/7c705cffb0334f09ae77de2c68776e4b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.734441 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b19b54-9870-48d7-bf9c-170fdab89fdb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.741423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.742335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.742547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.742907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.744276 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.746460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.768604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ccaae6b5064751928d74cf9b45b9b7, user_id=7c705cffb0334f09ae77de2c68776e4b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.771355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.798793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1228/2439] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:03 2022] PUT /identity/v3/projects/18ccaae6b5064751928d74cf9b45b9b7/users/7c705cffb0334f09ae77de2c68776e4b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:03.802475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.821287 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1212/2440] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:03.826863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:03.969469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdd62c-5fc1-41f0-868c-f730fb45db5e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.974794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.975572 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.975931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.976343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.977136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.978848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:03.979004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3078d6-a00e-4b5f-a7c4-53c05a3f26ad admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:03.982089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:03.989567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:03.990531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:03.991162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:03.991648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:03.992669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:03.994912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.004411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1229/2441] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:04.005366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.007850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.025082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1213/2442] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:03 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:04.027037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.031121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f015869a-4afd-4bac-a6d0-32a1f8141001 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.039491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608679a-bb52-43d2-81af-1fa2794108fa admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.043702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.045194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.045654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.046164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] PATH_INFO: `/v3/users/f735526cee0b4837862ad8e60277b7b3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.047660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f735526cee0b4837862ad8e60277b7b3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.051643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.052876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.054348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.054775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.055827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] PATH_INFO: `/v3/projects/18ccaae6b5064751928d74cf9b45b9b7/users/7c705cffb0334f09ae77de2c68776e4b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.064070 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ccaae6b5064751928d74cf9b45b9b7, user_id=7c705cffb0334f09ae77de2c68776e4b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.066985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.084725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1214/2443] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:04 2022] PUT /identity/v3/projects/18ccaae6b5064751928d74cf9b45b9b7/users/7c705cffb0334f09ae77de2c68776e4b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:04.085947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.116640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:04.120825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f735526cee0b4837862ad8e60277b7b3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:04.129466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f735526cee0b4837862ad8e60277b7b3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:04.130471 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f735526cee0b4837862ad8e60277b7b3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:04.145450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f735526cee0b4837862ad8e60277b7b3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:04.161296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-77be63b6-0119-4628-aacf-14de3437d26a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f735526cee0b4837862ad8e60277b7b3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:04.171218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1230/2444] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:04 2022] DELETE /identity/v3/users/f735526cee0b4837862ad8e60277b7b3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:04.171218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.171218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.171218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.172456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3c417-66f4-43da-a8d7-dae418625071 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.198559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] MFA Rules not processed for user `7c705cffb0334f09ae77de2c68776e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:04.199822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.239219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.240408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.240858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.241307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.242223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.244874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fb627-0a98-433a-9fd7-5081b37e321a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.262759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1215/2445] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:04.264616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.264972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.265319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.284383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1231/2446] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:04 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:04.285843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:04.287863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.289861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.291560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.303414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1216/2447] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:04.303414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.305610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.313032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6243e-62da-4b79-8ea0-98962776f62a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.323442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.324162 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.324519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.324733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.325649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.327142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.339081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1232/2448] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:04.341027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.344539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.395547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391349-ceba-4343-a71e-7fc83c06882e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.400469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.401522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.402213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.402663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.405653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:04.416831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.419511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.423941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81238054-ccf2-4aa5-9356-fc2b92df9026 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.428392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.429056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.429340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.429887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.434160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1217/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:04 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:04.435396 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.450130 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bde88-778e-4a23-a4c8-c0e121d7c0dc service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.461129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.462315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.463596 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.463766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.464036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] PATH_INFO: `/v3/projects/12dea4d3963c4d3485ba9ed95189a4ad/users/799c3fb373a74ec8a586d70ae230de15/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.467259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.499848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1233/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:04.515897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12dea4d3963c4d3485ba9ed95189a4ad, user_id=799c3fb373a74ec8a586d70ae230de15, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.526686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9cbfd-1270-40cf-83bf-bec735666587 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.565967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.571531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1218/2451] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:04 2022] PUT /identity/v3/projects/12dea4d3963c4d3485ba9ed95189a4ad/users/799c3fb373a74ec8a586d70ae230de15/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:04.578339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c7043-4819-4fc2-ac82-3e67822e8e9e service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:04.580647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.581038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.581384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.671748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] MFA Rules not processed for user `799c3fb373a74ec8a586d70ae230de15`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:04.675424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.717833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:04.718493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:04.720647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:04.720941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:04.724661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.741747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:04.847103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:04.859719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1219/2452] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:04.860646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:04.861082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:04.861931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:04.904940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1234/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:04.907874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.047358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339528b-0ff5-4ad8-944e-a576ec479bc0 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.053962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.055118 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.055453 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.055913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.056921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.061046 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.099872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1235/2454] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:04 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:05.102242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.105019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29287b80-b54c-4db4-b4e1-9e688d609ba1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.105558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.168081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:05.237885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.239242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.239799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.240354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] PATH_INFO: `/v3/users/36ac1fd0310c4eb8889a82628b2580a7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.242330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ac1fd0310c4eb8889a82628b2580a7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.246719 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.252702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b0a85-da7f-44b0-b028-869e791cadfd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.258579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.259558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.259842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.260133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] PATH_INFO: `/v3/projects/c7ddb21614eb44f4baeafacd2e3baead` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.261489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ddb21614eb44f4baeafacd2e3baead)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.264232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.327798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ac1fd0310c4eb8889a82628b2580a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.328987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ac1fd0310c4eb8889a82628b2580a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.329799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ac1fd0310c4eb8889a82628b2580a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.339708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ddb21614eb44f4baeafacd2e3baead', 'request_id': 'req-34dca020-6942-46e6-b695-b5ced98b8bee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.353743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ac1fd0310c4eb8889a82628b2580a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.358507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ddb21614eb44f4baeafacd2e3baead', 'request_id': 'req-34dca020-6942-46e6-b695-b5ced98b8bee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.375393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ddb21614eb44f4baeafacd2e3baead', 'request_id': 'req-34dca020-6942-46e6-b695-b5ced98b8bee'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.392539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ac1fd0310c4eb8889a82628b2580a7'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.399853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:05.413675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] The token cache is being invalidate because project c7ddb21614eb44f4baeafacd2e3baead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:05.414141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.414962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-34dca020-6942-46e6-b695-b5ced98b8bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:05.415996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1220/2455] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:04 2022] DELETE /identity/v3/users/36ac1fd0310c4eb8889a82628b2580a7 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:05.416947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1236/2456] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:05 2022] DELETE /identity/v3/projects/c7ddb21614eb44f4baeafacd2e3baead => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:05.417855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.421386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.422136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.422525 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.422915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.453984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:05.455523 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.523138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5187f55-1bdd-4d27-9790-61471cb5b3f3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.533803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.534507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.534794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.535074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.535918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.538209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72f431-6d3b-4e1b-a1ac-41219c1e8142 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.570857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1221/2457] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:05 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:05.576948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.577249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.577843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.578787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1237/2458] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:05.579755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.581213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.605591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:05.607494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.615855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135931f-77dc-43fe-bafe-939c8337ca94 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.626370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1222/2459] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:05.626370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.626370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.628807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.629971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.629971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.630329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.631527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.633101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5b086-6cfc-4383-bfc3-cd2d194726ab admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.650955 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1238/2460] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:05.652371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.659750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.660225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.701432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:05.706700 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:05.710562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9180d-c854-46d5-a5bf-2c84c3f42bcc None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.719180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.719849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.721852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.722137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.732382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1239/2461] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:05.733809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.754616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.760418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.761070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ad18c-1cac-4a72-a79f-503ff44cd139 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.768825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.769752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.770036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.770314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] PATH_INFO: `/v3/projects/fa8a7feeb0034db7836721c10bd4137a/users/ab2ec8acfe1e47a0a67e785041a7c3b6/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.793296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8a7feeb0034db7836721c10bd4137a, user_id=ab2ec8acfe1e47a0a67e785041a7c3b6, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.796478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.802903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1223/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:05.803946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.815792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1240/2463] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:05 2022] PUT /identity/v3/projects/fa8a7feeb0034db7836721c10bd4137a/users/ab2ec8acfe1e47a0a67e785041a7c3b6/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:05.816949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.838513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadb17-49df-4bfe-9095-9560b9807d24 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.842428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:05.916904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3be07e-c93d-4c53-94ce-3642638bfa73 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.925012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:05.925847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:05.926349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:05.926732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:05.927578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:05.929937 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:05.964428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1241/2464] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:05.974655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:05.988181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed509b-832c-4933-9e84-b8e0839bf02b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:05.999212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.001618 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.002216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.002782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] PATH_INFO: `/v3/projects/fa8a7feeb0034db7836721c10bd4137a/users/ab2ec8acfe1e47a0a67e785041a7c3b6/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.006526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa8a7feeb0034db7836721c10bd4137a, user_id=ab2ec8acfe1e47a0a67e785041a7c3b6, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.010717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327ed79d-f0d4-4729-af15-82ef3617efa7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.039068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1242/2465] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:05 2022] PUT /identity/v3/projects/fa8a7feeb0034db7836721c10bd4137a/users/ab2ec8acfe1e47a0a67e785041a7c3b6/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:06.040773 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.041168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.041551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.046947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.048138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.048452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.048781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.049628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:06.049935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.051731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.084655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] MFA Rules not processed for user `ab2ec8acfe1e47a0a67e785041a7c3b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:06.085527 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1224/2466] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:06.086444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:06.086815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.111828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e264c5-0fc6-42ce-9c56-7ececb3a12e0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.132833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:06.185810 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1243/2467] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:06.201750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.204361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:06.279552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.280268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.280558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.280843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.281555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.283499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.348213 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7d96a-f95e-400f-93cd-47519ca9ea01 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.351818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1225/2468] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:06 2022] POST /identity/v3/projects => generated 375 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:06.353093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.445475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:06.473632 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.476380 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.476582 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.476847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.506408 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf95c09-44b7-4109-adfa-68c531074e4c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.515760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.516618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.516825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.517158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.518263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.520404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.571651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1226/2469] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:06 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:06.573414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.588069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc65446-0c17-401f-aa43-d3d7c3cc01e1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.597077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.598169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.598689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.599197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.600588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.603683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.619097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1227/2470] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:06.625593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.642638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.644090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.659035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996ae43-5155-4d7e-a018-e0280dac09a5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.677580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.678412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.678709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.678996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.679750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.681916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1244/2471] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:06.695370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeee0a-0095-4a7b-9f30-df751d572bb8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.725262 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1228/2472] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:06.727493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:06.751676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffa9b6-f8b5-497f-9b4d-5f8111dd079b service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:06.767657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:06.768614 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.768904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.769201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] PATH_INFO: `/v3/projects/8e2fe3b3e7b042bd85e0a7775f2f35a1/users/2ea2011e97524b00bd471ffa19fcb24d/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.815288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e2fe3b3e7b042bd85e0a7775f2f35a1, user_id=2ea2011e97524b00bd471ffa19fcb24d, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:06.818220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:06.849227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1245/2473] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:06 2022] PUT /identity/v3/projects/8e2fe3b3e7b042bd85e0a7775f2f35a1/users/2ea2011e97524b00bd471ffa19fcb24d/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:06.861133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:06.862414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:06.863591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:06.952751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] MFA Rules not processed for user `2ea2011e97524b00bd471ffa19fcb24d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:06.957339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.020553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:07.022543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.149704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:07.175883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1229/2474] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:07.186232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:07.189093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.207524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852d771-6c10-4253-acb5-d66a15fb6785 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:07.219175 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:07.220067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:07.220383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:07.220661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] PATH_INFO: `/v3/projects/00239053d31046c493ca68ea15431fa3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:07.222244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00239053d31046c493ca68ea15431fa3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:07.225474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:07.382439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36c73-9314-44dd-9eb2-5ff3f7d641d4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:07.395727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00239053d31046c493ca68ea15431fa3', 'request_id': 'req-cfcb0139-d3ab-463a-bf3d-fc1297822d75'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.418210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00239053d31046c493ca68ea15431fa3', 'request_id': 'req-cfcb0139-d3ab-463a-bf3d-fc1297822d75'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.447055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00239053d31046c493ca68ea15431fa3', 'request_id': 'req-cfcb0139-d3ab-463a-bf3d-fc1297822d75'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.462643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:07.487131 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] The token cache is being invalidate because project 00239053d31046c493ca68ea15431fa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:07.487763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.489241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.491132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1246/2475] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:07 2022] DELETE /identity/v3/projects/00239053d31046c493ca68ea15431fa3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:07.495035 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:07.498138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.539065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:07.539726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:07.540021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:07.540338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:07.542217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.664477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcb0139-d3ab-463a-bf3d-fc1297822d75 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:07.672596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:07.673864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:07.674291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:07.674668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] PATH_INFO: `/v3/users/a7588aa439dc43449979fbb771ab531d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:07.676457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7588aa439dc43449979fbb771ab531d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:07.678402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:07.686348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:07.691871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:07.734286 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1230/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:07.735636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:07.747361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:07.793848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7588aa439dc43449979fbb771ab531d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.812298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7588aa439dc43449979fbb771ab531d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.812298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7588aa439dc43449979fbb771ab531d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.835518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7588aa439dc43449979fbb771ab531d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.843108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:07.868711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7588aa439dc43449979fbb771ab531d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:07.883106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1247/2477] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:07 2022] DELETE /identity/v3/users/a7588aa439dc43449979fbb771ab531d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:07.914426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543d0c2-1665-4222-927f-cf355903c434 service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.173125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab2ec8acfe1e47a0a67e785041a7c3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa8a7feeb0034db7836721c10bd4137a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.177809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.178092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.179372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.182583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:08.186173 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.194186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.223916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:08.225961 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:08.247610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1231/2478] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:08.251684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:08.257420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:08.304819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d549dc-4f26-446b-95af-b47f4023323b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.322514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab2ec8acfe1e47a0a67e785041a7c3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa8a7feeb0034db7836721c10bd4137a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.323131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.329484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.329910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.330685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.344580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.400799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1232/2479] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:08 2022] POST /identity/v3/projects => generated 354 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:08.400799 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:08.403221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:08.455163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e9195-54d1-4356-b8c0-1fd3a6dcbf43 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.500554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:08.589418 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.591058 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.591348 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.591624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] PATH_INFO: `/v3/users/2ea2011e97524b00bd471ffa19fcb24d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.593093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea2011e97524b00bd471ffa19fcb24d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.598320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.608982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11894486-c1f7-4917-a9e7-1fa99ffa93ae tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.619571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.620152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.620435 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.620743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] PATH_INFO: `/v3/users/b77f0d7681024db28eef64a86dd2aea2` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.630764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b77f0d7681024db28eef64a86dd2aea2)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.632792 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.709706 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea2011e97524b00bd471ffa19fcb24d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.731882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea2011e97524b00bd471ffa19fcb24d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.732642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea2011e97524b00bd471ffa19fcb24d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.746036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea2011e97524b00bd471ffa19fcb24d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.755263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b77f0d7681024db28eef64a86dd2aea2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.755986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b77f0d7681024db28eef64a86dd2aea2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.756681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b77f0d7681024db28eef64a86dd2aea2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.761012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:08.773703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b77f0d7681024db28eef64a86dd2aea2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.788074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea2011e97524b00bd471ffa19fcb24d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.798887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1248/2480] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:08 2022] DELETE /identity/v3/users/2ea2011e97524b00bd471ffa19fcb24d => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:08.799405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b77f0d7681024db28eef64a86dd2aea2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:08.800987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:08.803429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:08.817775 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1233/2481] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:08 2022] DELETE /identity/v3/users/b77f0d7681024db28eef64a86dd2aea2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:08.820787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:08.820911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:08.894783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd26d8-b7b2-4ce9-8411-2ab82e39c69d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.915229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff91028-9434-44b1-b7ba-9e50d32d397e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:08.920507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.921453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.921746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.922027 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] PATH_INFO: `/v3/projects/d047721f526e4940b6161807f0c0928b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.923297 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d047721f526e4940b6161807f0c0928b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.925747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.956521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:08.976015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab2ec8acfe1e47a0a67e785041a7c3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa8a7feeb0034db7836721c10bd4137a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:08.976828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:08.977460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:08.977921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:08.979267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:08.980887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:08.982829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c3836-7cb2-4e6e-aee4-b11b0925f609 tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:08.999844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1249/2482] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:09.002134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499dd7a-79a4-4cfe-90ae-2a513f858be3 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.002750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499dd7a-79a4-4cfe-90ae-2a513f858be3 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.003343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499dd7a-79a4-4cfe-90ae-2a513f858be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.020135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd047721f526e4940b6161807f0c0928b', 'request_id': 'req-c8fa38e7-c173-4d66-8b3b-b5242154f54d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.038452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7499dd7a-79a4-4cfe-90ae-2a513f858be3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:09.038946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd047721f526e4940b6161807f0c0928b', 'request_id': 'req-c8fa38e7-c173-4d66-8b3b-b5242154f54d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.039890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7499dd7a-79a4-4cfe-90ae-2a513f858be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.054466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd047721f526e4940b6161807f0c0928b', 'request_id': 'req-c8fa38e7-c173-4d66-8b3b-b5242154f54d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.078726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] The token cache is being invalidate because project d047721f526e4940b6161807f0c0928b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:09.079125 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.080201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c8fa38e7-c173-4d66-8b3b-b5242154f54d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.082574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1234/2483] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:08 2022] DELETE /identity/v3/projects/d047721f526e4940b6161807f0c0928b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:09.084921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee489-8c8f-4b86-af01-d47da648f1d6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.085337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee489-8c8f-4b86-af01-d47da648f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.087629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ee489-8c8f-4b86-af01-d47da648f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.138612 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-763ee489-8c8f-4b86-af01-d47da648f1d6 None None] MFA Rules not processed for user `a091e587a9c547cca66b526425afc2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:09.139986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-763ee489-8c8f-4b86-af01-d47da648f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.162292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1250/2484] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:09.164754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.165470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.166591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.197020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:09.200998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.218092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1251/2485] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:09.218092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.218581 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.227046 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:09.258285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1235/2486] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:09.259416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.259705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.259984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.285477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:09.286685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.300267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1236/2487] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:09.300267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.300267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.310057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33a5ad-6d76-466e-bdc2-8755723d68c6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.315905 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ab2ec8acfe1e47a0a67e785041a7c3b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa8a7feeb0034db7836721c10bd4137a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.316829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.317697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.317988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] PATH_INFO: `/v3/projects/793e221218134c37a0a31278831a88f3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.319983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorizing `identity:delete_project(project_id=793e221218134c37a0a31278831a88f3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.322127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.377326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da58de-ad34-4f99-a30a-ceba976cacde None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.382563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.383534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.384050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.389842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.404789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793e221218134c37a0a31278831a88f3', 'request_id': 'req-a6e6d0c4-7d4a-4732-85da-05616712609b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.412305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.414298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.417467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793e221218134c37a0a31278831a88f3', 'request_id': 'req-a6e6d0c4-7d4a-4732-85da-05616712609b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.429947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1237/2488] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:09.430912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.441386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398616-f025-41b9-8c1f-a3aaca2f86d9 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.450313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793e221218134c37a0a31278831a88f3', 'request_id': 'req-a6e6d0c4-7d4a-4732-85da-05616712609b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.475681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] The token cache is being invalidate because project 793e221218134c37a0a31278831a88f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:09.476056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.477094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:09.479133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1252/2489] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:09 2022] DELETE /identity/v3/projects/793e221218134c37a0a31278831a88f3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:09.480027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.481667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.516273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:09.523330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.524454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.525191 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.527455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.531978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.670938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.672283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c57c51-b206-4364-9845-a27b69cf58db service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.687859 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e6d0c4-7d4a-4732-85da-05616712609b tempest-VolumeQuotaClassesTest-503287474 tempest-VolumeQuotaClassesTest-503287474-project-admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.698524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.699659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.700102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.700401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.702257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1238/2490] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:09.703393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.708912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.709210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.709531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.757240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:09.758470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.778369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1239/2491] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:09.784762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.801404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6c7d5-4def-4a3a-ade1-ea6b6e5630b8 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.818639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.819406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.819595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.819877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.820650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:09.820958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.824181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.828467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.830141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.854490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1253/2492] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:09.861724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.864008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1240/2493] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:09.864908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:09.871345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:09.885226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4e081-17fc-40da-9960-64dff8967061 service glance] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.903086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.903902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.906801 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bc908-2794-470e-877b-6142a289c9e6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:09.907347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.907656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.908428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.910541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.928503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:09.929385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:09.929849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:09.930132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] PATH_INFO: `/v3/users/799c3fb373a74ec8a586d70ae230de15` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:09.931660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] RBAC: Authorizing `identity:delete_user(user_id=799c3fb373a74ec8a586d70ae230de15)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:09.935003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:09.948745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:09.964998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1254/2494] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:09.966159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.073443 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799c3fb373a74ec8a586d70ae230de15'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.074358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799c3fb373a74ec8a586d70ae230de15'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.075118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799c3fb373a74ec8a586d70ae230de15'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.080500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da824434-6678-4de2-9ad8-6cffffa53766 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.088487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.089484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.089964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.091055 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.092616 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799c3fb373a74ec8a586d70ae230de15'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.094732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.098715 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.118093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f9b2f8ac-f3df-4e89-aaeb-8ba858d7b58a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799c3fb373a74ec8a586d70ae230de15'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.139953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1241/2495] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:09 2022] DELETE /identity/v3/users/799c3fb373a74ec8a586d70ae230de15 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:10.153923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791d39-9d3e-491d-9616-d52f04a252a5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.161791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791d39-9d3e-491d-9616-d52f04a252a5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.162202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791d39-9d3e-491d-9616-d52f04a252a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.175202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1255/2496] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:09 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:10.180930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.184142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.210910 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83b3b0-ead2-4d11-a7de-d358c539afbb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.218744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:10.279995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c1791d39-9d3e-491d-9616-d52f04a252a5 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:10.281321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c1791d39-9d3e-491d-9616-d52f04a252a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.315420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.316346 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.316634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.317054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] PATH_INFO: `/v3/projects/8e2fe3b3e7b042bd85e0a7775f2f35a1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.318565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e2fe3b3e7b042bd85e0a7775f2f35a1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.323417 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.385811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e2fe3b3e7b042bd85e0a7775f2f35a1', 'request_id': 'req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.393252 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1242/2497] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:42:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:10.394763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.395209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.395364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.398123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e2fe3b3e7b042bd85e0a7775f2f35a1', 'request_id': 'req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.408092 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e2fe3b3e7b042bd85e0a7775f2f35a1', 'request_id': 'req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.421961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:10.423242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] The token cache is being invalidate because project 8e2fe3b3e7b042bd85e0a7775f2f35a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:10.423390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.423548 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.424349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.428204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1256/2498] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:10 2022] DELETE /identity/v3/projects/8e2fe3b3e7b042bd85e0a7775f2f35a1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:10.429071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.430551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.485684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:10.502721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018bacd-2b3e-4854-a3f4-cbb5144d9fc5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.507878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.508584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.508870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.509145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.509874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.510872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1243/2499] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:10.511627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbd361-ebb3-4543-bee3-768a872c7811 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.513970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.517694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.524699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1257/2500] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:10.526199 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.526494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.526776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.534015 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b1d48f-c4ff-4146-b5e6-eec2e62513ce None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.539343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.540234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.540521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.540800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] PATH_INFO: `/v3/projects/758a80f1cf56414582066891009ab9dd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.541996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758a80f1cf56414582066891009ab9dd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.544005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.548137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:10.549461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.564448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1258/2501] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:10.564448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.564448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.598096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:10.602609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758a80f1cf56414582066891009ab9dd', 'request_id': 'req-716ba707-bc41-46eb-82b2-c97512776a92'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.610188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758a80f1cf56414582066891009ab9dd', 'request_id': 'req-716ba707-bc41-46eb-82b2-c97512776a92'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.617781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758a80f1cf56414582066891009ab9dd', 'request_id': 'req-716ba707-bc41-46eb-82b2-c97512776a92'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.634525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] The token cache is being invalidate because project 758a80f1cf56414582066891009ab9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:10.634674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.635449 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.637825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1244/2502] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:10 2022] DELETE /identity/v3/projects/758a80f1cf56414582066891009ab9dd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:10.638605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.639907 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.673354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16c2683-652d-4518-80dc-12746839ede5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.712050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ba707-bc41-46eb-82b2-c97512776a92 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.716642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.717299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.717589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.717871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.719851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.751326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.751989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.752309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.752589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.755012 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.790759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.792134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.802217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1245/2503] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:10.803076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.804675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.816830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f81d3e-c084-4cfd-8df7-0373c525e183 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.821388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.822644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.822644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.822644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.823310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.825522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9478e-45a3-4c28-98cf-d32816ddd3f8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.827075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.829547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.839466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1246/2504] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:10.840985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.841346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.841685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.842137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1259/2505] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:42:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:42:10.843345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.844626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.860145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39826904-2a32-44b4-a966-bf8f7f02cdcb service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.870863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.872981 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.874533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.874836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] PATH_INFO: `/v3/users/7c705cffb0334f09ae77de2c68776e4b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.878302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c705cffb0334f09ae77de2c68776e4b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.882497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:10.899849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:10.901186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:10.917538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1247/2506] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:10.918636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:10.937066 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:10.946696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5dd861-4e50-49a9-90fb-954cd007fff3 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:10.965650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:10.966566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:10.966848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:10.967121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] PATH_INFO: `/v3/projects/f6710a9eccc94db88c4781ab8ba9998f/users/568811b139a34117921f255f9108ea02/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:10.980681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c705cffb0334f09ae77de2c68776e4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.982882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:10.993416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6710a9eccc94db88c4781ab8ba9998f, user_id=568811b139a34117921f255f9108ea02, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:10.994714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c705cffb0334f09ae77de2c68776e4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.995678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c705cffb0334f09ae77de2c68776e4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:10.996888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.011627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c705cffb0334f09ae77de2c68776e4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.019633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1248/2507] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:10 2022] PUT /identity/v3/projects/f6710a9eccc94db88c4781ab8ba9998f/users/568811b139a34117921f255f9108ea02/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:11.050647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.063571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9e308f55-48c2-438f-acba-755b745578eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c705cffb0334f09ae77de2c68776e4b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.081535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1260/2508] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:10 2022] DELETE /identity/v3/users/7c705cffb0334f09ae77de2c68776e4b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:11.081535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.081535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.081535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.147121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] MFA Rules not processed for user `568811b139a34117921f255f9108ea02`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:11.148622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.201672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da464d7b-ac26-4e52-8057-d10ed380542f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.286638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1261/2509] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:11.287727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.289747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.337693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.342325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.342625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.342915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.343672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:11.350609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.352351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.384895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1249/2510] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:11.385805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.392226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.411799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5b44c-764f-4027-99cf-5798e37c19b4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.418111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.419709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.419996 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.420309 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] PATH_INFO: `/v3/projects/12dea4d3963c4d3485ba9ed95189a4ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.421543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12dea4d3963c4d3485ba9ed95189a4ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.423646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.445686 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e97cb0-56a2-4dda-97c8-e5bbe7e40a7e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.458365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.460571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.461945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.463536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.465437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.472247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.537433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12dea4d3963c4d3485ba9ed95189a4ad', 'request_id': 'req-0b0898e3-d44e-4233-89c5-14515783b4a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.541251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1250/2511] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:11 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:11.542541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.545172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.559677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12dea4d3963c4d3485ba9ed95189a4ad', 'request_id': 'req-0b0898e3-d44e-4233-89c5-14515783b4a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.561315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:11.579669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12dea4d3963c4d3485ba9ed95189a4ad', 'request_id': 'req-0b0898e3-d44e-4233-89c5-14515783b4a6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.603515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] The token cache is being invalidate because project 12dea4d3963c4d3485ba9ed95189a4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:11.606769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.608270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.611716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1262/2512] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:11 2022] DELETE /identity/v3/projects/12dea4d3963c4d3485ba9ed95189a4ad => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:11.625244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.627802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.696398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:11.704866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248a147-afff-4157-9022-89c9fc66c774 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.713535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.714319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.714704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.715622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.716625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.719536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.780535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1251/2513] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:11 2022] POST /identity/v3/projects => generated 403 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:11.781813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.782806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0898e3-d44e-4233-89c5-14515783b4a6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.784973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.883825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab2297-d441-46c7-8ba3-32c402cf482a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:11.897044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.897873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.898162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.898701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] PATH_INFO: `/v3/projects/18ccaae6b5064751928d74cf9b45b9b7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.901219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ccaae6b5064751928d74cf9b45b9b7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.904859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.911089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:11.911696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:11.911986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:11.912301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:11.913007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:11.919414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:11.968663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1263/2514] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:11 2022] POST /identity/v3/users => generated 300 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:11.971360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:11.974499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:11.994999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ccaae6b5064751928d74cf9b45b9b7', 'request_id': 'req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:11.999376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3bb09-1374-4d92-9de1-256bbeb88122 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.015838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ccaae6b5064751928d74cf9b45b9b7', 'request_id': 'req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.037478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ccaae6b5064751928d74cf9b45b9b7', 'request_id': 'req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.069873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.071722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.071722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.071722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] PATH_INFO: `/v3/users/ab2ec8acfe1e47a0a67e785041a7c3b6` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.073789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2ec8acfe1e47a0a67e785041a7c3b6)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.077051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.088503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] The token cache is being invalidate because project 18ccaae6b5064751928d74cf9b45b9b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:12.088795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.091730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.093495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1252/2515] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:11 2022] DELETE /identity/v3/projects/18ccaae6b5064751928d74cf9b45b9b7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:12.097759 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.102472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.131034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a5f63-2d3f-4c04-8dbf-ab86e48d04e2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.134272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:12.141421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.143369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.146771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.147466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.148587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.154788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.191646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2ec8acfe1e47a0a67e785041a7c3b6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.203861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2ec8acfe1e47a0a67e785041a7c3b6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.205630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2ec8acfe1e47a0a67e785041a7c3b6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.206634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1253/2516] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:12 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:12.207582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.209528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.226860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02fea-3853-468a-8fad-5ed3760e5ec8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.234133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2ec8acfe1e47a0a67e785041a7c3b6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.239321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.240677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.240989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.241277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.242037 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.243696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909a51e-d8d9-4793-be80-74ff64976a9f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.271451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1254/2517] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:12.273357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.273880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.275000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.294166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2ec8acfe1e47a0a67e785041a7c3b6'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:12.301466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:12.303067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.311831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1264/2518] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:11 2022] DELETE /identity/v3/users/ab2ec8acfe1e47a0a67e785041a7c3b6 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:12.312812 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.316823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.408936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:12.435895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03189ea3-947e-4a42-81dc-854839c3cb8c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.440895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.441652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.445431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1255/2519] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:12.449285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.450369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.451239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.453403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3756f-8c01-4170-948e-aad5ccec9c7b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.457143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.469399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.472757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1265/2520] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:12.472757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.472757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.472757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.487693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29927c59-0159-460e-aa76-9a85b1f4581d None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.501448 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:12.502882 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.507197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.508132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.508423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.508711 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] PATH_INFO: `/v3/projects/c6bff57a65924d1fa04e29fe50a02546/users/5b60bda429954b149b51cca9aa6db603/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.519134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1266/2521] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:12.519988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.541542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28507bef-6181-474c-a752-d53631c8cda0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.545048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bff57a65924d1fa04e29fe50a02546, user_id=5b60bda429954b149b51cca9aa6db603, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.547743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.547884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.548749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.549129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.549563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.550689 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.552848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.570637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1267/2522] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:12.571752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.576957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1256/2523] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:12 2022] PUT /identity/v3/projects/c6bff57a65924d1fa04e29fe50a02546/users/5b60bda429954b149b51cca9aa6db603/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:12.577935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.584529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf225194-2408-46bb-a0eb-892a5d444268 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.657774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.658538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.659031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.659508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.660952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:12.660952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.663130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.677853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1268/2524] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:12.678825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.692051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fa6ce-527a-448f-9341-88ff98f61bec admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.698219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.698804 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.699220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.699713 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.699971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb175d2-398e-40f1-bbcc-0543ea816942 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.700783 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.702949 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.711296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.712456 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.713013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.713465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] PATH_INFO: `/v3/projects/14c668ca95f441de9bcc5c4bb1095ffe/users/60274912d3ec404abe5d345d78224c52/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.716357 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1269/2525] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:12.719787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.725452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:12.736947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b121ea3-0e79-48d3-a50c-a887b1d4a7be admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.746329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.747965 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.748451 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.749279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.750356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.753580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.775056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14c668ca95f441de9bcc5c4bb1095ffe, user_id=60274912d3ec404abe5d345d78224c52, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.778240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.792282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1270/2526] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:12 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:12.793704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.816514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1257/2527] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:12 2022] PUT /identity/v3/projects/14c668ca95f441de9bcc5c4bb1095ffe/users/60274912d3ec404abe5d345d78224c52/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:12.816514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:12.826723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:12.877990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d5494-c184-4c3e-8e7d-b6d654acd7a5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.951797 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.952241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff9e7f-64c6-405b-a45d-08f715b81781 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:12.952733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.953024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.953310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] PATH_INFO: `/v3/projects/c6bff57a65924d1fa04e29fe50a02546/users/5b60bda429954b149b51cca9aa6db603/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.955333 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bff57a65924d1fa04e29fe50a02546, user_id=5b60bda429954b149b51cca9aa6db603, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.957925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:12.958103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.958986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:12.959440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:12.959830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] PATH_INFO: `/v3/projects/fa8a7feeb0034db7836721c10bd4137a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:12.961159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa8a7feeb0034db7836721c10bd4137a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:12.963278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:12.970436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1271/2528] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:12 2022] PUT /identity/v3/projects/c6bff57a65924d1fa04e29fe50a02546/users/5b60bda429954b149b51cca9aa6db603/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:12.971268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.070437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6205c-4968-4cc7-bffe-da7615bc5fd9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.082825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:13.094291 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa8a7feeb0034db7836721c10bd4137a', 'request_id': 'req-65041c8e-438c-4e97-a9f4-1023c04d4de5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:13.118639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:13.166534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa8a7feeb0034db7836721c10bd4137a', 'request_id': 'req-65041c8e-438c-4e97-a9f4-1023c04d4de5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:13.181083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa8a7feeb0034db7836721c10bd4137a', 'request_id': 'req-65041c8e-438c-4e97-a9f4-1023c04d4de5'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:13.184751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.186005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.186422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.186705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.188142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.190353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20887-7c88-4946-b9e4-b94090d4a41c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.201427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] The token cache is being invalidate because project fa8a7feeb0034db7836721c10bd4137a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:13.201985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:13.206881 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-65041c8e-438c-4e97-a9f4-1023c04d4de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:13.214915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1258/2529] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:12 2022] DELETE /identity/v3/projects/fa8a7feeb0034db7836721c10bd4137a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:13.216731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489242bc-c60e-4bad-8e9f-7173a844b6cf None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.217118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489242bc-c60e-4bad-8e9f-7173a844b6cf None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.217502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489242bc-c60e-4bad-8e9f-7173a844b6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.249517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1272/2530] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:12 2022] POST /identity/v3/users => generated 285 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:13.249755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-489242bc-c60e-4bad-8e9f-7173a844b6cf None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:13.249871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240beb71-6023-433d-8a99-d69eae3c1c6a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.249871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240beb71-6023-433d-8a99-d69eae3c1c6a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.250062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240beb71-6023-433d-8a99-d69eae3c1c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.250831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-489242bc-c60e-4bad-8e9f-7173a844b6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.276603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1259/2531] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:13.280489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37990578-db20-449c-8944-783dcc302cf9 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.280866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37990578-db20-449c-8944-783dcc302cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.281219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37990578-db20-449c-8944-783dcc302cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.286982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-240beb71-6023-433d-8a99-d69eae3c1c6a None None] MFA Rules not processed for user `5b60bda429954b149b51cca9aa6db603`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:13.288279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-240beb71-6023-433d-8a99-d69eae3c1c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.321282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-37990578-db20-449c-8944-783dcc302cf9 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:13.322983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-37990578-db20-449c-8944-783dcc302cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.333455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1260/2532] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:13.334486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37990578-db20-449c-8944-783dcc302cf9 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.342255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-37990578-db20-449c-8944-783dcc302cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.360475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37990578-db20-449c-8944-783dcc302cf9 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.372761 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.373826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.375355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.375355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.375355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.377036 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.397742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1261/2533] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:13.398635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.400723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.432778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1273/2534] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:13.441566 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.442452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.442935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.469973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:13.471626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.495033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1274/2535] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:13.495033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.505998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd556a7-3936-49c9-8c6f-804a62ab25e0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.512981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217d530-b9d5-4f73-8406-b43f4fbb2886 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.513166 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.514039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.514488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.514839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.516147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.517178 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.517827 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.517952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.518117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.518404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.529481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.530949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.533568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1275/2536] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:13.534233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.536071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.547246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1262/2537] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:13.548143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.557982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a0dd0-9441-4cfc-97e2-ff2a6de34322 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.564741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.565782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.566451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.566949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.568296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:13.568947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.570806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.593831 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1263/2538] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:13.596092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.609960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86047660-4afe-450f-9d04-831a3851e496 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.617893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.618950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.619361 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.619781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] PATH_INFO: `/v3/projects/bfe08af614304d06b940c6293bcba4f0/users/86188fa4d9f34ac999fecde0427cb50b/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.657943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe08af614304d06b940c6293bcba4f0, user_id=86188fa4d9f34ac999fecde0427cb50b, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.658275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4812d39-f0c4-4738-bc15-58f9dcbcb1ce admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.663580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.664765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.665514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.665994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.678080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.684677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.687444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c034399-b273-4fbb-8ffa-48888e01741c service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.723184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1264/2539] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:13 2022] PUT /identity/v3/projects/bfe08af614304d06b940c6293bcba4f0/users/86188fa4d9f34ac999fecde0427cb50b/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:13.724213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:13.727927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1276/2540] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:13.740044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.740611 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.741350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.820865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] MFA Rules not processed for user `86188fa4d9f34ac999fecde0427cb50b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:13.824308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:13.900803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:13.934826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da0890-f96c-4954-9002-aad1d6c780d5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:13.943111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:13.944692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:13.945209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:13.945709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:13.946658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:13.949059 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:13.992581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:14.015724 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1265/2541] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/projects => generated 379 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:14.016805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.018678 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.033462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1277/2542] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:14.033462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.066752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb43b8c-a596-4923-ad3e-f53f62a4e9f2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.173734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be7cc3-c42c-4aa3-862e-ea648bc600e9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.195319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.199735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.200172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.203603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] PATH_INFO: `/v3/users/568811b139a34117921f255f9108ea02` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.204953 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=568811b139a34117921f255f9108ea02)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.207148 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.260671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.263405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.263805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.264214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.267273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.273255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.334252 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1278/2543] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:14 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:14.340709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.360503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '568811b139a34117921f255f9108ea02'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:14.362057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '568811b139a34117921f255f9108ea02'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:14.363010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '568811b139a34117921f255f9108ea02'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:14.387524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '568811b139a34117921f255f9108ea02'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:14.389741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfb4dcc-5a6b-4fca-ab8f-e3e0071fe50c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.406057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.406886 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.407274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.407660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.408514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.411262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdedec9-6552-4027-9a37-5a398b49fd7f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.414288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '568811b139a34117921f255f9108ea02'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:14.448849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1266/2544] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:14 2022] DELETE /identity/v3/users/568811b139a34117921f255f9108ea02 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:14.450812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.454832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.465112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1279/2545] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:14 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:14.469082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.469442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.469780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.513452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:14.514837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.597357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab04d3-c6a6-4222-8316-870df0a4a8ad admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.626529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:14.664757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1280/2546] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:14.671143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.675306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.678386 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.679197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.679995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.679995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.680849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.682713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.694415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c871a-6979-42dd-b01e-ef9e7a841a98 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.703577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1267/2547] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:14.704844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.708976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.710113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.710302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.710583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.710860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.713829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.716378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.767284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1281/2548] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:14 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:14.770549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.785899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dfe42b-d848-4c70-93ca-8079b0638b5c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.791449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.792212 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.792646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.792935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.794253 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.797680 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.803883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:14.825487 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1282/2549] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:14.826986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.841510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e953999-c739-4208-b473-93d1a91dad7d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.842407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431cb51-8ac2-44f1-ad8e-f24cc7595eca admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.847796 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.848510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.848696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.849094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.849476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.849901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.855686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.856871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.858192 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.861034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94edd7e-9e9c-4a3d-ae87-db8dcadef7b0 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.864684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:14.867425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:14.885377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1283/2550] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:14.891733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068197-9592-4839-8327-6dccb95f9d33 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.893050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068197-9592-4839-8327-6dccb95f9d33 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.893463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d068197-9592-4839-8327-6dccb95f9d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:14.907773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1268/2551] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:14.923557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:14.950882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1da23-e76b-4543-ad75-92fdf1551a0b service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:14.956794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3d068197-9592-4839-8327-6dccb95f9d33 None None] MFA Rules not processed for user `60274912d3ec404abe5d345d78224c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:14.958789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3d068197-9592-4839-8327-6dccb95f9d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:14.970911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:14.971811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:14.972090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:14.972440 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] PATH_INFO: `/v3/projects/68e9a5a8805a4914b7dc5304343ab4ea/users/0be015354a534162a736e8fa980d53a3/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.015056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e9a5a8805a4914b7dc5304343ab4ea, user_id=0be015354a534162a736e8fa980d53a3, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.018497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.045048 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1269/2552] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:14 2022] PUT /identity/v3/projects/68e9a5a8805a4914b7dc5304343ab4ea/users/0be015354a534162a736e8fa980d53a3/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:15.048306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.125942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1284/2553] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:15.127430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.127737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.128007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.174649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d8012-c3a5-4451-bef7-2a7e3fb03fb5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.183060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.185625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.186664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.187189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] PATH_INFO: `/v3/projects/f0d28c348433406aaf5a3db9e748de12/users/47977f2a697746f4921dfb392894c037/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.194874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] MFA Rules not processed for user `0be015354a534162a736e8fa980d53a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:15.196811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:15.208480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d28c348433406aaf5a3db9e748de12, user_id=47977f2a697746f4921dfb392894c037, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.212439 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.267007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1270/2554] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:15 2022] PUT /identity/v3/projects/f0d28c348433406aaf5a3db9e748de12/users/47977f2a697746f4921dfb392894c037/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:15.270910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.275515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:15.320017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1285/2555] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:15.321560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.330050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:15.340410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5970a57e-187a-4a56-ac38-0b3edc464164 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.399876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fd8ba-d0f6-4e2d-94cc-33a70b1f0805 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.406893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.407569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.407851 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.408149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.413495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.415434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.456339 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1271/2556] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/projects => generated 403 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:15.457391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.502957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.507934 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.508662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.509171 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.511378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.513441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.534076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1286/2557] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:15.534076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.551555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:15.560853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb6dfe-5d61-4dd5-8705-f852916607d3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.574446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cf6bc-2cc0-48db-a3af-374b8fdfe2d2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.579587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.580461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.580988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.581367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.582162 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.584186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.638071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1272/2558] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:15.639003 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.646995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.647677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.647964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.648277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.648988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.651241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.676714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1287/2559] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:15.677785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.694136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da32088-03c7-473d-97bd-fa1e0caa42ff admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.731817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124265-0861-47c3-8e65-cbb7369f8e66 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.737990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.738901 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.739190 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.739477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] PATH_INFO: `/v3/projects/f0d28c348433406aaf5a3db9e748de12/users/47977f2a697746f4921dfb392894c037/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.742399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0d28c348433406aaf5a3db9e748de12, user_id=47977f2a697746f4921dfb392894c037, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.744982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.760451 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1273/2560] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:15 2022] PUT /identity/v3/projects/f0d28c348433406aaf5a3db9e748de12/users/47977f2a697746f4921dfb392894c037/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:15.761607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.808227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.808492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.808602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.809143 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.810076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.811526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.824916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:15.837321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1288/2561] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:15.837321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.841037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:15.854328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c5647-0e75-44cc-a290-e0e35673eb99 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.859088 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.860028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.860411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.862620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] PATH_INFO: `/v3/projects/f6710a9eccc94db88c4781ab8ba9998f` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.862620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6710a9eccc94db88c4781ab8ba9998f)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.864753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.865452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faaefa-6123-47d3-904a-04dec68ba001 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:15.879097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:15.879908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.880461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.880876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.882384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:15.884392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f04292-7dd2-4300-8524-83fa6892fd1f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:15.898017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1274/2562] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:15.899521 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:15.899817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:15.900099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:15.930187 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6710a9eccc94db88c4781ab8ba9998f', 'request_id': 'req-837d2d16-dd7c-4f11-986d-b112df38238a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:15.930513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] MFA Rules not processed for user `47977f2a697746f4921dfb392894c037`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:15.931751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:15.943746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6710a9eccc94db88c4781ab8ba9998f', 'request_id': 'req-837d2d16-dd7c-4f11-986d-b112df38238a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:15.951136 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6710a9eccc94db88c4781ab8ba9998f', 'request_id': 'req-837d2d16-dd7c-4f11-986d-b112df38238a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:15.966664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] The token cache is being invalidate because project f6710a9eccc94db88c4781ab8ba9998f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:15.966954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:15.967686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:15.969761 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1289/2563] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:15 2022] DELETE /identity/v3/projects/f6710a9eccc94db88c4781ab8ba9998f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:15.970462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:15.971883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:15.982531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:16.041788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1275/2564] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:16.043731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.045856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.080801 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d2d16-dd7c-4f11-986d-b112df38238a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.086030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.086723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.087006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.087283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.088939 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.090855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.105530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1290/2565] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:16.105530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.108178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.122229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189af281-362f-4dea-b0d5-9e4e7276e4bc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.128277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.129185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.129474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.129753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] PATH_INFO: `/v3/users/5b60bda429954b149b51cca9aa6db603` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.131115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b60bda429954b149b51cca9aa6db603)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.133116 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.157665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef3c6a-396f-4ea7-990a-ba79814a4959 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.168189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.169494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.170033 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.170446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] PATH_INFO: `/v3/projects/4890af8aae374d90acca42fe21013193/users/67e40011f9a844498a5310ac0d489ca2/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.194173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b60bda429954b149b51cca9aa6db603'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:16.203209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b60bda429954b149b51cca9aa6db603'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:16.203912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b60bda429954b149b51cca9aa6db603'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:16.208693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4890af8aae374d90acca42fe21013193, user_id=67e40011f9a844498a5310ac0d489ca2, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.216640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.228048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b60bda429954b149b51cca9aa6db603'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:16.246950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1276/2566] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:16 2022] PUT /identity/v3/projects/4890af8aae374d90acca42fe21013193/users/67e40011f9a844498a5310ac0d489ca2/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:16.246950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.248527 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b60bda429954b149b51cca9aa6db603'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:16.249805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.260028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1291/2567] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:16 2022] DELETE /identity/v3/users/5b60bda429954b149b51cca9aa6db603 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:16.261745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.274660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.325479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:16.389999 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af54ac9-c6ee-4d5b-ae72-e2ad15f3e756 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.412984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ec512-1f2b-4a70-9705-6334fa170b54 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.428185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.431206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.433030 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.435204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.438610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.443950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.463138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.464062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.464488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.464871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.466299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1292/2568] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:16.468017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.472903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.477407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.525364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2467b75-07a8-41c7-bd29-fd46e118fc14 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.541075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.542511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.543383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.543878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.545975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.554289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.555225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:16.561283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.564505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.579463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1277/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:16 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:16.580865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.591865 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1293/2570] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:16.597625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.597797 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b417732-be60-4d25-9002-4cfeeecb627f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.606277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.608427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.609265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.609677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] PATH_INFO: `/v3/projects/b364ec89880d4fbfbbf83f005f6995ae/users/a23cfbf7c8424dfaa9f3241f8d1f40fd/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.624959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de4ad9-e315-4f5b-860d-81f7a3ea2b1c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.646811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b364ec89880d4fbfbbf83f005f6995ae, user_id=a23cfbf7c8424dfaa9f3241f8d1f40fd, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.650906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85a139-5b75-4680-94a0-98f2e5d9397c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.653458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.654657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.655169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.655671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] PATH_INFO: `/v3/projects/4890af8aae374d90acca42fe21013193/users/67e40011f9a844498a5310ac0d489ca2/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.658176 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4890af8aae374d90acca42fe21013193, user_id=67e40011f9a844498a5310ac0d489ca2, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.661291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.678512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1294/2571] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:16 2022] PUT /identity/v3/projects/4890af8aae374d90acca42fe21013193/users/67e40011f9a844498a5310ac0d489ca2/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:16.680373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1278/2572] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:16 2022] PUT /identity/v3/projects/b364ec89880d4fbfbbf83f005f6995ae/users/a23cfbf7c8424dfaa9f3241f8d1f40fd/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:16.680737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.687853 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.700762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.701195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.711032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.765295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] MFA Rules not processed for user `a23cfbf7c8424dfaa9f3241f8d1f40fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:16.768042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.785744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908e04f-5a33-4ec4-9369-0fea8587f6f6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:16.796362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:16.797991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:16.800093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:16.800431 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:16.803896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.928080 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1279/2573] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:16.929444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:16.932312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:16.942837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:16.950155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:16.954468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:16.970120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1295/2574] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:16.971105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.100463 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801a795-939b-4057-a662-20061fd4acf6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.114427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.115319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.115610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.115890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] PATH_INFO: `/v3/projects/c6bff57a65924d1fa04e29fe50a02546` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.124199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6bff57a65924d1fa04e29fe50a02546)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.127077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.183054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b6c83-c176-4822-af10-718cce641962 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.201373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.203936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.205344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.206867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.209110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.216691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.239781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6bff57a65924d1fa04e29fe50a02546', 'request_id': 'req-0ce15b77-3145-474c-83ed-85627fe15e26'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:17.254926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6bff57a65924d1fa04e29fe50a02546', 'request_id': 'req-0ce15b77-3145-474c-83ed-85627fe15e26'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:17.267922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6bff57a65924d1fa04e29fe50a02546', 'request_id': 'req-0ce15b77-3145-474c-83ed-85627fe15e26'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:17.299106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] The token cache is being invalidate because project c6bff57a65924d1fa04e29fe50a02546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:17.299373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:17.301950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0ce15b77-3145-474c-83ed-85627fe15e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:17.303917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1280/2575] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:16 2022] DELETE /identity/v3/projects/c6bff57a65924d1fa04e29fe50a02546 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:17.317244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdd10-aff9-44a2-8c2f-d6e4d34f02c5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.317543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdd10-aff9-44a2-8c2f-d6e4d34f02c5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.317835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cdd10-aff9-44a2-8c2f-d6e4d34f02c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.337059 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1296/2576] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:16 2022] POST /identity/v3/projects => generated 379 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:17.344658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.347299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:17.364818 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-452cdd10-aff9-44a2-8c2f-d6e4d34f02c5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:17.367028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-452cdd10-aff9-44a2-8c2f-d6e4d34f02c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:17.406191 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:17.476285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dade82-6367-4bd3-a0c7-c73b5b78cb95 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.485789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.486876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.487362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.487830 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.489832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.493658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.519863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1281/2577] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:17.539901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1297/2578] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/users => generated 296 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:17.544987 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.545837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.546154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.546447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.559796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56950b1e-8744-4999-a56c-b8d143f6a9c6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.565148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.566101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.566385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.566718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.567577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.569922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.585813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1298/2579] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:17.586876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.648733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:17.651220 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4b939-d7a4-4e82-995f-b24660d00c06 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.654387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:17.673415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.674924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.675476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.675906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] PATH_INFO: `/v3/projects/472bc6114c13460a91a7dd687b3a64fd/users/8c6f31b4d0c64ed287abe3e8cf74b747/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.683473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1282/2580] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:17.690001 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.707820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e01e-3d9d-4c6c-9b04-3953310f9931 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.710146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=472bc6114c13460a91a7dd687b3a64fd, user_id=8c6f31b4d0c64ed287abe3e8cf74b747, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.713659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747f786-c17c-4e7a-b957-60e0506e2095 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.731565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.732295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.732636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.732903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.733894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:17.734213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.735989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.747079 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1299/2581] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:17 2022] PUT /identity/v3/projects/472bc6114c13460a91a7dd687b3a64fd/users/8c6f31b4d0c64ed287abe3e8cf74b747/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:17.749550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.749995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.750423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.754628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1283/2582] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:17.754628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.766810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53db561-be6a-4ad9-b48d-141d1f85a9f3 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.781084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:17.792403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] MFA Rules not processed for user `67e40011f9a844498a5310ac0d489ca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:17.797325 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:17.839309 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.840553 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.840845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.841121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.841871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.844774 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.856964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1284/2583] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:17.857806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.868675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b31be-0d84-4e14-aa09-1f6623eae28b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:17.873658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:17.874634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:17.874921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:17.875196 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:17.875887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:17.877963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:17.913420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1300/2584] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:17.913420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.915434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1285/2585] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:17.920789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:17.951002 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226888e7-2bd9-4540-954b-e1edd087f416 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.083729 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b290b62-d79c-4355-8252-9b4ac1840964 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.090988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.091900 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.092206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.092485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] PATH_INFO: `/v3/projects/472bc6114c13460a91a7dd687b3a64fd/users/8c6f31b4d0c64ed287abe3e8cf74b747/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.097904 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] RBAC: Authorizing `identity:create_grant(project_id=472bc6114c13460a91a7dd687b3a64fd, user_id=8c6f31b4d0c64ed287abe3e8cf74b747, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.100861 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.101543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.101879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.102158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.102912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.105228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.106183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8284a4-7d72-4b4a-b28a-97e30066992f admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.119427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:18.128154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1301/2586] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:17 2022] PUT /identity/v3/projects/472bc6114c13460a91a7dd687b3a64fd/users/8c6f31b4d0c64ed287abe3e8cf74b747/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:18.141943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.142237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.143422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.153664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1286/2587] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:17 2022] POST /identity/v3/users => generated 293 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:18.156063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.188434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e14615-48b7-4ce7-a4d3-843868835643 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.209434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] MFA Rules not processed for user `8c6f31b4d0c64ed287abe3e8cf74b747`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:18.210655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:18.293778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.294814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.295538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.295887 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.298172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.305773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.319206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1302/2588] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:18.327654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.329442 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:18.335663 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1287/2589] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:18.340662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.353089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65790a19-1502-4de9-b92c-7cbd8a58a414 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.360869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.363406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.363406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.363406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.363406 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.364745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.384398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1288/2590] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:42:18.393065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.407357 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:18.411879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a92e73-5520-4bab-99dc-9b29d86b8ef1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.422023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c62a6d-b65f-4add-8888-585b0609b77b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.425108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.426090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.426228 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.426399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.426802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.427013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] PATH_INFO: `/v3/projects/eb3d302ea5164552b8b6fbf0a2ddaff6/users/5772e50f7a5a4ed0ab5e13a1191c5f5e/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.427117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.427361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.443787 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.445267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ceff9-fc2e-4e3e-b98b-d9fb1cab5eec service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.452370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3d302ea5164552b8b6fbf0a2ddaff6, user_id=5772e50f7a5a4ed0ab5e13a1191c5f5e, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.456492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.457077 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1303/2591] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:18.476161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1289/2592] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:18 2022] PUT /identity/v3/projects/eb3d302ea5164552b8b6fbf0a2ddaff6/users/5772e50f7a5a4ed0ab5e13a1191c5f5e/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:18.486524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.486813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.489070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.514954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.516942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:18.548179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] MFA Rules not processed for user `5772e50f7a5a4ed0ab5e13a1191c5f5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:18.552238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:18.584856 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd894-b9e2-42b4-80ef-9749fa15632c admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.589258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.589948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.590238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.590524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.592675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:18.661241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.662840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.671375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1304/2593] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:18.673307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1290/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:18.676805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.741100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:18.829512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855712-c4cf-4eec-b34c-5ef16353d3af None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:18.835200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:18.835943 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:18.836326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:18.836678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:18.837637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:18.839593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:18.889652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1305/2595] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:18 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:18.893683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:18.910557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769e65e-8cfb-4f7d-9604-45fe530ddf9f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.065972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.066972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.067273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.067607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.068524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.071000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.106996 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1291/2596] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:18 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:19.112627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.130029 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a102fc-c771-4999-9f77-302f40c06545 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.142674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.144577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.148128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.148419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.149195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.152206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.179927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1306/2597] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:19.183909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.200957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f528-3c5a-4f93-9dd0-17b6e568edef admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.210780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.211965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.212462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.212905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] PATH_INFO: `/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9/users/b4bb430a857f4ad1a105f0a4be345b3d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.262768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eacb3dd8504bb1b6e665c08b0417f9, user_id=b4bb430a857f4ad1a105f0a4be345b3d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.272120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.301373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:19.318892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1292/2598] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:19 2022] PUT /identity/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9/users/b4bb430a857f4ad1a105f0a4be345b3d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:19.324591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.337661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b3e0-2623-405d-9dd6-0ccea54a134a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.485007 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.485765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.486139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.486979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.487564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.492688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f9515-5694-4008-86f8-cc3b2538e743 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.510692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1307/2599] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:19.515823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.536738 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbad8a-02e3-41b6-ac10-dd5e093f4117 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.553584 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.554646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.554940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.555369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] PATH_INFO: `/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9/users/b4bb430a857f4ad1a105f0a4be345b3d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.558749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9eacb3dd8504bb1b6e665c08b0417f9, user_id=b4bb430a857f4ad1a105f0a4be345b3d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.561838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.584973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1293/2600] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:19 2022] PUT /identity/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9/users/b4bb430a857f4ad1a105f0a4be345b3d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:19.587895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.588589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.588589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.608529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.610564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:19.641056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] MFA Rules not processed for user `b4bb430a857f4ad1a105f0a4be345b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:19.642286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:19.720582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd43c5a-aadf-417c-b78b-4f3064405bee admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.727444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.728699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.729053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.729634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] PATH_INFO: `/v3/users/60274912d3ec404abe5d345d78224c52` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.731620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60274912d3ec404abe5d345d78224c52)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.734589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.758936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1308/2601] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:19.771322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:19.792847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:19.807017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60274912d3ec404abe5d345d78224c52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:19.807906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60274912d3ec404abe5d345d78224c52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:19.809209 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60274912d3ec404abe5d345d78224c52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:19.820171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60274912d3ec404abe5d345d78224c52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:19.840789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60274912d3ec404abe5d345d78224c52'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:19.852668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f9393-f8d3-4c09-851c-197bf8e10a38 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:19.856748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:19.857422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:19.857681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:19.857980 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:19.859827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:19.860693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1294/2602] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:19 2022] DELETE /identity/v3/users/60274912d3ec404abe5d345d78224c52 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:19.896010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:19.897717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:19.913291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1309/2603] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:20.085593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:20.090030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:20.184716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7faa24-da76-4091-b885-71b13c828fe4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:20.189773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:20.191084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:20.191084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:20.191337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] PATH_INFO: `/v3/projects/14c668ca95f441de9bcc5c4bb1095ffe` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:20.192575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14c668ca95f441de9bcc5c4bb1095ffe)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:20.196587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:20.216161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:20.276453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14c668ca95f441de9bcc5c4bb1095ffe', 'request_id': 'req-c0261c6f-2e8f-4455-8872-335aafbf5931'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.282673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14c668ca95f441de9bcc5c4bb1095ffe', 'request_id': 'req-c0261c6f-2e8f-4455-8872-335aafbf5931'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.294137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14c668ca95f441de9bcc5c4bb1095ffe', 'request_id': 'req-c0261c6f-2e8f-4455-8872-335aafbf5931'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.311050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] The token cache is being invalidate because project 14c668ca95f441de9bcc5c4bb1095ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:20.311050 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.311706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.317414 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1295/2604] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:20 2022] DELETE /identity/v3/projects/14c668ca95f441de9bcc5c4bb1095ffe => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:20.318427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:20.329936 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf1818-5057-4b84-9bd0-41541d2c4685 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:20.428836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:20.435412 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:20.435827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:20.436646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] PATH_INFO: `/v3/users/67e40011f9a844498a5310ac0d489ca2` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:20.439130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e40011f9a844498a5310ac0d489ca2)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:20.441338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:20.531913 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e40011f9a844498a5310ac0d489ca2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.546150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e40011f9a844498a5310ac0d489ca2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.546150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e40011f9a844498a5310ac0d489ca2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.557305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e40011f9a844498a5310ac0d489ca2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.573919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e40011f9a844498a5310ac0d489ca2'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:20.583195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1310/2605] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:20 2022] DELETE /identity/v3/users/67e40011f9a844498a5310ac0d489ca2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:20.813893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:20.822062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:20.850455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0261c6f-2e8f-4455-8872-335aafbf5931 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:20.913874 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:20.919828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.007245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.008461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.008828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.009555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] PATH_INFO: `/v3/users/5772e50f7a5a4ed0ab5e13a1191c5f5e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.035415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5772e50f7a5a4ed0ab5e13a1191c5f5e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.038637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.040766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:21.127003 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be2217-f543-4b0e-92b7-9170ba57a77d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.142696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.143872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.145742 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.148995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] PATH_INFO: `/v3/projects/4890af8aae374d90acca42fe21013193` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.151681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4890af8aae374d90acca42fe21013193)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.152486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5772e50f7a5a4ed0ab5e13a1191c5f5e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.153617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5772e50f7a5a4ed0ab5e13a1191c5f5e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.153930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.155271 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5772e50f7a5a4ed0ab5e13a1191c5f5e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.169081 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5772e50f7a5a4ed0ab5e13a1191c5f5e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.196858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-42117ec5-1128-45aa-bb14-7ce2898c3d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5772e50f7a5a4ed0ab5e13a1191c5f5e'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.222030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1296/2606] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:20 2022] DELETE /identity/v3/users/5772e50f7a5a4ed0ab5e13a1191c5f5e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:21.224447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.224816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.225181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.257006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:21.257494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4890af8aae374d90acca42fe21013193', 'request_id': 'req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.258843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.267941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4890af8aae374d90acca42fe21013193', 'request_id': 'req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.273884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:21.281692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4890af8aae374d90acca42fe21013193', 'request_id': 'req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.305379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] The token cache is being invalidate because project 4890af8aae374d90acca42fe21013193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:21.306040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.307184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-3fdb3e92-fd32-44f3-84ea-4783ff34c6c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.310122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1311/2607] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:20 2022] DELETE /identity/v3/projects/4890af8aae374d90acca42fe21013193 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:21.316558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224cbf-d4f7-4950-999f-eb018c8ad420 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.317154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224cbf-d4f7-4950-999f-eb018c8ad420 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.317623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224cbf-d4f7-4950-999f-eb018c8ad420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.343775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-07224cbf-d4f7-4950-999f-eb018c8ad420 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:21.346367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-07224cbf-d4f7-4950-999f-eb018c8ad420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.400375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1297/2608] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:21.411968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.413858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.433042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1312/2609] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:21.441932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.442578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.443691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.476083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:21.477714 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.493359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1313/2610] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:21.498278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.505007 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513b7663-f04a-4885-9f5e-5a3a45b49ef7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.510043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21dd6ba-3bb7-4539-943e-14e3c754afe1 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.510311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.511074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.511372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.511746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.515750 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.518022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.518315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.518594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.520432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:21.521285 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.524730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.524947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.526806 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.542094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1314/2611] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:21.544692 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1298/2612] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:21.546968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.566825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011b3d5-d0fc-43ca-acf2-c829f02cfa7b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.573807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.575332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.575791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.576421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.577294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.579596 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.601646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.603239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.623015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1315/2613] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:21.623518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780a21f-2024-401f-b294-8e86c8e63655 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.627524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.676367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:21.736283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.737237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.737652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.738006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] PATH_INFO: `/v3/users/86188fa4d9f34ac999fecde0427cb50b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.738236 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed28215-5b3e-47aa-8e91-a903137b9a4c admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.739649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=86188fa4d9f34ac999fecde0427cb50b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.742118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.756629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.757334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.757624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.757909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.758622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.761436 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.801539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1316/2614] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:21.805496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.819964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86188fa4d9f34ac999fecde0427cb50b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.820825 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86188fa4d9f34ac999fecde0427cb50b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.822354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86188fa4d9f34ac999fecde0427cb50b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.825897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aee80-70f8-40be-b31e-b8ff796466bd admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.833683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.834359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.834674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.836018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86188fa4d9f34ac999fecde0427cb50b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.837514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.842206 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:21.843860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:21.849399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:21.855706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-3bfb55c9-fc65-4437-b803-6e8de7c9e0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86188fa4d9f34ac999fecde0427cb50b'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:21.880121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1317/2615] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:21.881182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:21.884866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.888062 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1299/2616] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:21 2022] DELETE /identity/v3/users/86188fa4d9f34ac999fecde0427cb50b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:21.892911 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:21.893213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:21.893501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:21.906298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ac8ad-dd31-43a7-81c5-4e6861dfbbcb admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:21.927486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:21.928875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:21.993401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:21.994684 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.008690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.009091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.010407 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.013308 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.027056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1300/2617] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:22.028051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.029460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.033682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1318/2618] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:42:22.034629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.036608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.041601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031582bb-fe91-45b8-aaa9-bcb0843ae0db None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.046915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.047802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.048220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.048497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] PATH_INFO: `/v3/projects/eb3d302ea5164552b8b6fbf0a2ddaff6` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.049781 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3d302ea5164552b8b6fbf0a2ddaff6)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.052906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.110579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df2e14-705e-475c-a540-8ef6803872f2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.115242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3d302ea5164552b8b6fbf0a2ddaff6', 'request_id': 'req-7e653ddf-af1b-4397-aa42-f2506f55b2cb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.123127 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3d302ea5164552b8b6fbf0a2ddaff6', 'request_id': 'req-7e653ddf-af1b-4397-aa42-f2506f55b2cb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.133214 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3d302ea5164552b8b6fbf0a2ddaff6', 'request_id': 'req-7e653ddf-af1b-4397-aa42-f2506f55b2cb'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.152244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] The token cache is being invalidate because project eb3d302ea5164552b8b6fbf0a2ddaff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:22.152546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.153486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.159043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1301/2619] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:22 2022] DELETE /identity/v3/projects/eb3d302ea5164552b8b6fbf0a2ddaff6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:22.159043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.160865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.181730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.182381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.182663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.182990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.188001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.194646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:22.273779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.275388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.286968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1319/2620] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:22.287832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.290161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.297749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e653ddf-af1b-4397-aa42-f2506f55b2cb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.305513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148d10e-2d2a-4fa5-8780-8581ce3e0730 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.319889 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.320600 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.320890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.321178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] PATH_INFO: `/v3/users/b4bb430a857f4ad1a105f0a4be345b3d` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.322433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4bb430a857f4ad1a105f0a4be345b3d)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.325193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.327853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.328925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.329211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.329498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] PATH_INFO: `/v3/projects/9bbf2f328485486eba70f5e50e564897/users/a18382e32fd5486e8ef7fd0d48979c3a/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.333460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:22.354478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bbf2f328485486eba70f5e50e564897, user_id=a18382e32fd5486e8ef7fd0d48979c3a, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.357878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.381391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1302/2621] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:22 2022] PUT /identity/v3/projects/9bbf2f328485486eba70f5e50e564897/users/a18382e32fd5486e8ef7fd0d48979c3a/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:22.382221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.383895 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.423230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4bb430a857f4ad1a105f0a4be345b3d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.438599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4bb430a857f4ad1a105f0a4be345b3d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.439653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4bb430a857f4ad1a105f0a4be345b3d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.454984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4bb430a857f4ad1a105f0a4be345b3d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.487196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a4c9b1fb-b85b-43e3-ba5b-d0d055fafa51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4bb430a857f4ad1a105f0a4be345b3d'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.495573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1320/2622] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:22 2022] DELETE /identity/v3/users/b4bb430a857f4ad1a105f0a4be345b3d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:22.508051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c605760-2829-455e-a246-f95c09f5360f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.508369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c605760-2829-455e-a246-f95c09f5360f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.508690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c605760-2829-455e-a246-f95c09f5360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.514914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb88e16-edff-48c4-b13e-8c678248bebb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.569998 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9c605760-2829-455e-a246-f95c09f5360f None None] MFA Rules not processed for user `a18382e32fd5486e8ef7fd0d48979c3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:22.571495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9c605760-2829-455e-a246-f95c09f5360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.627249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.629035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.629646 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.630150 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] PATH_INFO: `/v3/users/0be015354a534162a736e8fa980d53a3` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.632218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be015354a534162a736e8fa980d53a3)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.634946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.680780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1321/2623] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:22 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:22.686547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c605760-2829-455e-a246-f95c09f5360f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.688096 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9c605760-2829-455e-a246-f95c09f5360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.703885 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:22.712728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c605760-2829-455e-a246-f95c09f5360f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.735129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.739477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.740221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.740825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.742457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.747201 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.776018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be015354a534162a736e8fa980d53a3'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.778496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be015354a534162a736e8fa980d53a3'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.780393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be015354a534162a736e8fa980d53a3'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.804095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be015354a534162a736e8fa980d53a3'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.823483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be015354a534162a736e8fa980d53a3'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:22.824254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1322/2624] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:22.826432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.829039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.841622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1303/2625] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:22 2022] DELETE /identity/v3/users/0be015354a534162a736e8fa980d53a3 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:22.841622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:22.842281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:22.861388 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:22.937206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3304fa-0329-441c-acd3-043d4a1b8ce4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.943914 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.944657 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.944935 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.945218 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.946945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.950540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaa39d-42a4-48ed-9894-72231f7495ac admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.959682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0823a-49e5-4ba8-85ce-5321453e2131 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:22.966642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:22.968495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.969062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.969769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] PATH_INFO: `/v3/projects/bfe08af614304d06b940c6293bcba4f0` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:22.972824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe08af614304d06b940c6293bcba4f0)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:22.975490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:22.990779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1304/2626] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:22 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:22.992503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:22.992864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:22.993155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.017240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:23.018520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.029598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1305/2627] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:23.033691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.034918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.044716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe08af614304d06b940c6293bcba4f0', 'request_id': 'req-a65f9155-f976-4feb-95ab-1004bf4578b9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.047495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1662d-cea8-42ef-bbe5-a7f1ec3f57e5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.055122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe08af614304d06b940c6293bcba4f0', 'request_id': 'req-a65f9155-f976-4feb-95ab-1004bf4578b9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.063803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe08af614304d06b940c6293bcba4f0', 'request_id': 'req-a65f9155-f976-4feb-95ab-1004bf4578b9'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.080536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] The token cache is being invalidate because project bfe08af614304d06b940c6293bcba4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:23.080746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.082430 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.084570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1323/2628] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:22 2022] DELETE /identity/v3/projects/bfe08af614304d06b940c6293bcba4f0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:23.085707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.087314 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.138637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.139793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.140074 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.140375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] PATH_INFO: `/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.141810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9eacb3dd8504bb1b6e665c08b0417f9)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.143920 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.179449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f9155-f976-4feb-95ab-1004bf4578b9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.186427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.189637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.190067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.190509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.191513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.193275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.205494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1324/2629] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:23.206583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9eacb3dd8504bb1b6e665c08b0417f9', 'request_id': 'req-e80d5596-4265-40b5-af47-a9bd8ac8714f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.209603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.211535 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.213548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9eacb3dd8504bb1b6e665c08b0417f9', 'request_id': 'req-e80d5596-4265-40b5-af47-a9bd8ac8714f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.222385 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9eacb3dd8504bb1b6e665c08b0417f9', 'request_id': 'req-e80d5596-4265-40b5-af47-a9bd8ac8714f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.225871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:23.243654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] The token cache is being invalidate because project e9eacb3dd8504bb1b6e665c08b0417f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:23.244404 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.245445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.252373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1306/2630] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:23 2022] DELETE /identity/v3/projects/e9eacb3dd8504bb1b6e665c08b0417f9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:23.253325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.257160 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.289251 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:23.307347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade015b-6d18-43e2-9a92-e873e0f22098 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.334005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d5596-4265-40b5-af47-a9bd8ac8714f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.342412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.343442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.343908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.344318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] PATH_INFO: `/v3/users/47977f2a697746f4921dfb392894c037` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.345873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47977f2a697746f4921dfb392894c037)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.348419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.367156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.367818 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.368101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.368401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.370241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.383538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.385013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.397278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1325/2631] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:23.397928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.399248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.411877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb47577-6c74-4031-89e1-9450a4db7062 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.417363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47977f2a697746f4921dfb392894c037'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.417511 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.419092 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47977f2a697746f4921dfb392894c037'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.419258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.419458 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.419728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] PATH_INFO: `/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1/users/6aabbcbbfade40d996db9081ba41264d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.421172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47977f2a697746f4921dfb392894c037'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.435258 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47977f2a697746f4921dfb392894c037'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.439685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfef34a3e9b44df683d8baaefdbf81c1, user_id=6aabbcbbfade40d996db9081ba41264d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.442445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f22952a-a3ba-43e2-8ccc-feaf4aa986ad admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.453057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47977f2a697746f4921dfb392894c037'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:23.465332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1326/2632] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:23 2022] PUT /identity/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1/users/6aabbcbbfade40d996db9081ba41264d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:23.466717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6018c-2d59-493f-9905-0855a87e9d52 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.467010 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6018c-2d59-493f-9905-0855a87e9d52 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.467286 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6018c-2d59-493f-9905-0855a87e9d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.473096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1307/2633] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:23 2022] DELETE /identity/v3/users/47977f2a697746f4921dfb392894c037 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:23.474466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.477016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.491970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-06b6018c-2d59-493f-9905-0855a87e9d52 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:23.493647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06b6018c-2d59-493f-9905-0855a87e9d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.570871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cad65d-c09c-4dd7-a1a1-7cd88a0ca16f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.590195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1327/2634] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:23.590535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.590535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.590535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.590535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.590937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.590937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.592036 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.592469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.592832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.607177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1308/2635] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:23.611143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.628384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:23.629698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.630189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dfe5e-52f5-484c-8a59-38ba1dc338ad admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.636882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.638372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.638958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.639483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] PATH_INFO: `/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1/users/6aabbcbbfade40d996db9081ba41264d/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.644388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1328/2636] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:23.645534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfef34a3e9b44df683d8baaefdbf81c1, user_id=6aabbcbbfade40d996db9081ba41264d, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.645730 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.650822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ba239-2675-4f16-afcc-61e46a3e8062 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.658477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bcd20-4ad4-4e68-a0d9-c7052ed2b917 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.676708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.677450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.677736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.678013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.678842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:23.679050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.684381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1309/2637] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:23 2022] PUT /identity/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1/users/6aabbcbbfade40d996db9081ba41264d/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:23.685588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.694893 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.697900 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.698203 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.750795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1329/2638] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:23.755669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.775006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6862c-ce63-407b-89ed-772985fb0687 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:23.795589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] MFA Rules not processed for user `6aabbcbbfade40d996db9081ba41264d`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:23.797468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.808269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:23.859150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:23.859940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:23.860484 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:23.860871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:23.862105 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:23.863726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:23.878043 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:23.934509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1330/2639] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:23 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:23.935411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:23.939922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:23.986188 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1310/2640] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:23 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:23.986669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.017518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4965f-9824-490a-b36d-d1a393bdf516 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.088292 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ccf45-d86e-4145-927d-2d28fecf3867 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.094477 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.095344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.095777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.096465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] PATH_INFO: `/v3/projects/68e9a5a8805a4914b7dc5304343ab4ea` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.097690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e9a5a8805a4914b7dc5304343ab4ea)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.100677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.153072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.153816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.154102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.154387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.155170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.157089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.168459 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e9a5a8805a4914b7dc5304343ab4ea', 'request_id': 'req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.181580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e9a5a8805a4914b7dc5304343ab4ea', 'request_id': 'req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.184272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1311/2641] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:23 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:24.186095 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.189000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.189234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e9a5a8805a4914b7dc5304343ab4ea', 'request_id': 'req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.203779 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] The token cache is being invalidate because project 68e9a5a8805a4914b7dc5304343ab4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:24.204091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.204946 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.207914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1331/2642] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:23 2022] DELETE /identity/v3/projects/68e9a5a8805a4914b7dc5304343ab4ea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:24.208743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.210089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.260903 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6307e32-e2a9-4e3e-a4a4-1510daf01d01 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.271821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:24.299716 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca5bc0-a559-4da9-9d30-8605c5b34fff admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.305099 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.306273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.306738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.307039 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] PATH_INFO: `/v3/projects/f0d28c348433406aaf5a3db9e748de12` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.308291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0d28c348433406aaf5a3db9e748de12)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.310247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.326528 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.327331 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.327721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.328145 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.330335 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.350636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:24.377184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0d28c348433406aaf5a3db9e748de12', 'request_id': 'req-2cc88185-73ab-4b11-9be0-fd849f11769e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.393610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0d28c348433406aaf5a3db9e748de12', 'request_id': 'req-2cc88185-73ab-4b11-9be0-fd849f11769e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.404500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0d28c348433406aaf5a3db9e748de12', 'request_id': 'req-2cc88185-73ab-4b11-9be0-fd849f11769e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.416869 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] The token cache is being invalidate because project f0d28c348433406aaf5a3db9e748de12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:24.417336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.418754 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.423794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1332/2643] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:24 2022] DELETE /identity/v3/projects/f0d28c348433406aaf5a3db9e748de12 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:24.424897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.426267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.426464 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.427795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.444816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1312/2644] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:24.446110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.447798 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.468834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8afd4-1207-4f0b-ba4b-905d0f1401f8 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.533352 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc88185-73ab-4b11-9be0-fd849f11769e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.538179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.538975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.539329 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.539678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.540557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.542219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fd83f0-b030-407d-9fef-be77299cd37a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.557663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1333/2645] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:24.559227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.559540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.559895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.562299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.563181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.563464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.563869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] PATH_INFO: `/v3/users/a23cfbf7c8424dfaa9f3241f8d1f40fd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.565124 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23cfbf7c8424dfaa9f3241f8d1f40fd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.567255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.580733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:24.582170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.595260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1334/2646] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:24.595850 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.601301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.632951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23cfbf7c8424dfaa9f3241f8d1f40fd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.633805 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23cfbf7c8424dfaa9f3241f8d1f40fd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.634490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23cfbf7c8424dfaa9f3241f8d1f40fd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.647548 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23cfbf7c8424dfaa9f3241f8d1f40fd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.667347 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23cfbf7c8424dfaa9f3241f8d1f40fd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:24.677992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827851c-0781-40ab-809a-290e1afda2dc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.681994 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.682470 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1313/2647] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:24 2022] DELETE /identity/v3/users/a23cfbf7c8424dfaa9f3241f8d1f40fd => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:24.685504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.685504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.685750 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.686413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.686413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.698321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.703016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:24.771399 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:24.800408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.802042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ba877-c962-4825-8bb0-e5ff579d2bf0 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.812732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1335/2648] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:24.813834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.814122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.814399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.837095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:24.837519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e3a64-ac32-4978-919b-d51b07cddcfc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.839215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:24.846376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.846376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.846376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.846807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.847536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.851813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1336/2649] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:24.851977 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.856347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.867371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89269d83-f73e-459b-94fd-6a280a7eee0f None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.872492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1314/2650] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:24.874210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.876897 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.877820 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.878279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.878654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.881306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:24.881747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.887245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.889497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc664b84-e88c-4e3e-9633-ed77b3d0993a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.898864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.901088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.901432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.901720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] PATH_INFO: `/v3/projects/e31fe158a72c4028b8ebd143c79e09b5/users/5b05ad24e50549fc9930b1ded8593a3c/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.908544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1337/2651] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:42:24.912877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:24.927960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11959ad8-eaf6-40d4-893c-fdfa2aae12d9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:24.936644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:24.937274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:24.937555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:24.937834 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:24.938530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.940664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.958618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31fe158a72c4028b8ebd143c79e09b5, user_id=5b05ad24e50549fc9930b1ded8593a3c, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:24.962745 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cefa7b-4bd6-49f6-9552-04166690d104 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:24.988622 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1338/2652] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:24 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:24.988872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1315/2653] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:24 2022] PUT /identity/v3/projects/e31fe158a72c4028b8ebd143c79e09b5/users/5b05ad24e50549fc9930b1ded8593a3c/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:25.001569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.002844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.003184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.003681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.035961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] MFA Rules not processed for user `5b05ad24e50549fc9930b1ded8593a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:25.037605 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.092015 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e000d6-2bea-4b70-a1b1-46ee3a53950a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.100181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.101551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.102607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.103128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.104152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.106699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.140997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1339/2654] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:25 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:25.142513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.145534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.163968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0967c8c-c618-45b8-a8cb-495a0438f054 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.166071 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1316/2655] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:25.167225 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.176147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.176823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.177250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.177686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] PATH_INFO: `/v3/projects/b364ec89880d4fbfbbf83f005f6995ae` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.179073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b364ec89880d4fbfbbf83f005f6995ae)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.181392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf0818-e25f-4dab-a333-cd15110519dd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.181570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.191526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.194032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.194497 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.196539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.197411 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.201815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.218327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1317/2656] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:25.218327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.221933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.255305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b364ec89880d4fbfbbf83f005f6995ae', 'request_id': 'req-10136b49-ee7a-4a00-9066-d42ee05cf177'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.266446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b364ec89880d4fbfbbf83f005f6995ae', 'request_id': 'req-10136b49-ee7a-4a00-9066-d42ee05cf177'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.270648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:25.279752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b364ec89880d4fbfbbf83f005f6995ae', 'request_id': 'req-10136b49-ee7a-4a00-9066-d42ee05cf177'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.309074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] The token cache is being invalidate because project b364ec89880d4fbfbbf83f005f6995ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:25.309560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.310321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe8406-70c9-412b-b9c5-c1add01e0805 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.310463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.312479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1340/2657] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:25 2022] DELETE /identity/v3/projects/b364ec89880d4fbfbbf83f005f6995ae => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:25.313647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.315428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.351005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:25.375018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.375685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.375961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.376274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.378295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.395486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10136b49-ee7a-4a00-9066-d42ee05cf177 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.401381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.402857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.403294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.403578 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.404898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.407616 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.425767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1341/2658] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:25.426847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.430669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.451005 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e96cec-b404-49e9-886e-323aa3894bb7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.456989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.457891 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.458169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.458452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] PATH_INFO: `/v3/users/8c6f31b4d0c64ed287abe3e8cf74b747` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.459642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6f31b4d0c64ed287abe3e8cf74b747)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.461687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.470342 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.471948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.486523 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1318/2659] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:25.487625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.501770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9138ee-ced2-40a7-af82-d40bc677bf96 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.512426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.513547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.513933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.514345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] PATH_INFO: `/v3/projects/e86d133adc4c480d9b4982af8e585e90/users/90a5f32842f54f029bf4e8c212ee5a64/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.535702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6f31b4d0c64ed287abe3e8cf74b747'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.537580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86d133adc4c480d9b4982af8e585e90, user_id=90a5f32842f54f029bf4e8c212ee5a64, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.541838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6c868-6362-4c2a-bc3f-8a211ab12099 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.546855 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6f31b4d0c64ed287abe3e8cf74b747'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.547733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6f31b4d0c64ed287abe3e8cf74b747'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.562220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1319/2660] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:25 2022] PUT /identity/v3/projects/e86d133adc4c480d9b4982af8e585e90/users/90a5f32842f54f029bf4e8c212ee5a64/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:25.567013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6f31b4d0c64ed287abe3e8cf74b747'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.575057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.575467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.575848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.604155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6f31b4d0c64ed287abe3e8cf74b747'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:25.613704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1342/2661] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:25 2022] DELETE /identity/v3/users/8c6f31b4d0c64ed287abe3e8cf74b747 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:25.617096 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] MFA Rules not processed for user `90a5f32842f54f029bf4e8c212ee5a64`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:25.618316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.724905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1320/2662] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:25.743433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.745654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:25.819720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:25.833368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7700-ce46-47ce-9e10-e2829b756426 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.840650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:25.841660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:25.842428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:25.842757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:25.843723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:25.845911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:25.882289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1343/2663] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:25 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:25.887184 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:25.907238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333716f7-1754-4f49-b9c9-69c902aa62a1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:25.987076 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:26.006269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.006886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.008041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.008041 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.008381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.013249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.055679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1321/2664] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:25 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:26.061825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.079318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96239cc0-703d-4799-a2b8-e3fbcde99ae5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.085272 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.086186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.086654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.087247 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.088315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.090127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.114328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1344/2665] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:26.118912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.132362 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d145029-27a7-49a6-8b77-448ea054f016 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.137145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.140547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.144084 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.145995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.146364 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.146720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] PATH_INFO: `/v3/projects/7c0f2f367daf4a668d83232db352f75b/users/e0c63200a4594b23930feec36a01cf9f/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.169718 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0f2f367daf4a668d83232db352f75b, user_id=e0c63200a4594b23930feec36a01cf9f, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.172855 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.179476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74052-20c3-4e16-9483-b397737bca8b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.192488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.193798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.194124 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.194526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] PATH_INFO: `/v3/projects/472bc6114c13460a91a7dd687b3a64fd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.195935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=472bc6114c13460a91a7dd687b3a64fd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.198094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.201158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1322/2666] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:26 2022] PUT /identity/v3/projects/7c0f2f367daf4a668d83232db352f75b/users/e0c63200a4594b23930feec36a01cf9f/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:26.202099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.268876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472bc6114c13460a91a7dd687b3a64fd', 'request_id': 'req-701c2945-7808-4e9f-a15d-375e90029bbf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:26.279906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472bc6114c13460a91a7dd687b3a64fd', 'request_id': 'req-701c2945-7808-4e9f-a15d-375e90029bbf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:26.290662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472bc6114c13460a91a7dd687b3a64fd', 'request_id': 'req-701c2945-7808-4e9f-a15d-375e90029bbf'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:26.296154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cbffa-435d-4e42-b816-1f8b5bf446a9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.310727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] The token cache is being invalidate because project 472bc6114c13460a91a7dd687b3a64fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:26.311354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:26.312169 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-701c2945-7808-4e9f-a15d-375e90029bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:26.314851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1345/2667] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:26 2022] DELETE /identity/v3/projects/472bc6114c13460a91a7dd687b3a64fd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:26.320646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02050-0f69-4a00-93f8-32d9060b8e32 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.322681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02050-0f69-4a00-93f8-32d9060b8e32 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.322681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b02050-0f69-4a00-93f8-32d9060b8e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.345347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f9b02050-0f69-4a00-93f8-32d9060b8e32 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:26.346584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f9b02050-0f69-4a00-93f8-32d9060b8e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.393341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.395201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.395617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.396279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.398183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.400204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.416708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1323/2668] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:26.421696 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.424824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.443500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf29bcb-8a98-4b33-ae42-2cf29c317529 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.448329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.448748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1346/2669] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:26.449469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.449821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.450169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] PATH_INFO: `/v3/projects/7c0f2f367daf4a668d83232db352f75b/users/e0c63200a4594b23930feec36a01cf9f/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.452288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0f2f367daf4a668d83232db352f75b, user_id=e0c63200a4594b23930feec36a01cf9f, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.455119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3e598-f2dd-4b85-bdf0-84fd0544f5c2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.456908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.457379 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.459415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.466824 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1324/2670] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:26 2022] PUT /identity/v3/projects/7c0f2f367daf4a668d83232db352f75b/users/e0c63200a4594b23930feec36a01cf9f/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:26.482437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:26.483808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.532937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.533412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.533603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.570283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:26.585382 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] MFA Rules not processed for user `e0c63200a4594b23930feec36a01cf9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:26.586885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.611870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1347/2671] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:26.612964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.634619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eed80-fec6-4eb0-af82-15c573221381 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.645531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.646216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.646504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.646781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.647537 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:26.647844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.649492 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.678926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:26.681756 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1348/2672] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:26.687028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.710507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfbe45-17fc-44ec-99d1-04982a60f788 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.723647 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.724344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.725300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.725588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.730817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.732895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.748338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1325/2673] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:26.758426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.760659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.774375 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1349/2674] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/projects => generated 381 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:26.775320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.828559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34067127-691e-4222-a6f1-5be8d4b336b6 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.839330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6523bc-1d3b-42f4-b130-1014b30ffcb3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.844726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.845772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.845772 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.846014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.846790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.848732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddecd8-9ced-4290-bd4a-8e02de34b301 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.869454 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1350/2675] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:26.870798 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.871149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.871489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.882011 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.882680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.882959 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.883232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.889216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] MFA Rules not processed for user `eda8b1053f95458abfd8ea79227a0dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:26.890467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:26.946029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.946347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1351/2676] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:26.947119 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:26.949963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.958159 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21d087-9a70-48a3-8412-68f3db0ae7ce None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:26.963115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1326/2677] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:26.963269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:26.964185 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:26.965239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:26.965595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:26.966395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:26.968061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:26.982617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1352/2678] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:26.987224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.003514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf73103-0304-4943-b877-07353f4b9348 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.009093 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.009850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.010201 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.011946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.013152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.014854 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.030938 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1327/2679] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:27.033210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.033534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.043637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771ec2-8ca3-4ae8-86ae-35630031b499 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.045272 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad8b07-bf78-47f4-89e6-319bdd9710b5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.049133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.050024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.050307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.050587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] PATH_INFO: `/v3/projects/69ec3273b5d640809f9264b3fcd4675e/users/404cb9c8286d45748ef0ae1371e3973e/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.052078 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.053650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.053940 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.054223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] PATH_INFO: `/v3/users/90a5f32842f54f029bf4e8c212ee5a64` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.056304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a5f32842f54f029bf4e8c212ee5a64)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.058232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.070074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ec3273b5d640809f9264b3fcd4675e, user_id=404cb9c8286d45748ef0ae1371e3973e, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.073503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852df44-1f18-4115-baca-4728363afee5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.084934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:27.091075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1353/2680] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:27 2022] PUT /identity/v3/projects/69ec3273b5d640809f9264b3fcd4675e/users/404cb9c8286d45748ef0ae1371e3973e/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:27.124687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a5f32842f54f029bf4e8c212ee5a64'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:27.125874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a5f32842f54f029bf4e8c212ee5a64'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:27.126422 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a5f32842f54f029bf4e8c212ee5a64'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:27.138878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a5f32842f54f029bf4e8c212ee5a64'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:27.160128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.160608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.160734 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.166319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-84c589d6-7c52-4613-a6d9-9d824b87bbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a5f32842f54f029bf4e8c212ee5a64'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:27.184513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1328/2681] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:27 2022] DELETE /identity/v3/users/90a5f32842f54f029bf4e8c212ee5a64 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:27.189842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.190136 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.190415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.198800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] MFA Rules not processed for user `404cb9c8286d45748ef0ae1371e3973e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:27.200919 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.206924 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:27.215492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:27.219372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.289445 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1354/2682] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:27.290289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.291694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.299248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1329/2683] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:27.300349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.303782 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.318302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209f4d1-7b4e-406a-afee-a69f1c3435f5 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.326082 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.326741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.327028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.327387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.328606 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.331087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.361943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1330/2684] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:27 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:27.361943 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.365115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5db4a9-df26-4364-a80d-6fc10da9f859 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.417546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.418229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.418515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.418803 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.420767 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.426822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c47-f1a1-4c71-acb0-ff630871ee0f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.430517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.431151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.431433 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.431708 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.436343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.438107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.447661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1355/2685] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:27.448328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.475090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:27.526158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.527701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.534699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aec4b5-b465-41bd-9390-1ff1327db7a7 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.541033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.541758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.542045 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.542324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.543426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.545441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.555078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:27.567807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1331/2686] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:27.576973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1356/2687] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:27 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:27.581261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.591762 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b202d-dafe-43a1-86d0-40beef49d6c9 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.598321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.599073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.599423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.599882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.600726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.602734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.617746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1332/2688] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:27.622791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.636485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cc9dd-9a69-46e9-bba2-615910d36f30 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.641974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.642956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.643338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.643708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] PATH_INFO: `/v3/projects/1da724350bb646d4b80c8536079213c8/users/5f54d176941a40e2a6affdaeeaea1b9c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.659414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da724350bb646d4b80c8536079213c8, user_id=5f54d176941a40e2a6affdaeeaea1b9c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.662709 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.680920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1357/2689] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:27 2022] PUT /identity/v3/projects/1da724350bb646d4b80c8536079213c8/users/5f54d176941a40e2a6affdaeeaea1b9c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:27.683993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.713126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41625556-17d7-4384-a0c7-b31548fc1045 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.812835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.813591 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.813949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.814238 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.815265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.817142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6f013-61a4-4450-96e9-50eb805c1f5d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.834434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1333/2690] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:27.838927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.852829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fbc2-f6ea-4ffc-b1f0-982d7077de74 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:27.861324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:27.865758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.866259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.866649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] PATH_INFO: `/v3/projects/1da724350bb646d4b80c8536079213c8/users/5f54d176941a40e2a6affdaeeaea1b9c/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.869377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da724350bb646d4b80c8536079213c8, user_id=5f54d176941a40e2a6affdaeeaea1b9c, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:27.872976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:27.888457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1358/2691] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:27 2022] PUT /identity/v3/projects/1da724350bb646d4b80c8536079213c8/users/5f54d176941a40e2a6affdaeeaea1b9c/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:27.898773 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:27.899452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:27.899904 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:27.920938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:27.924958 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:27.937627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] MFA Rules not processed for user `5f54d176941a40e2a6affdaeeaea1b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:27.942194 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.053983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347d51c-265c-4e7b-abc0-3745f0d115f4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.059952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:28.060639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:28.060968 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:28.061424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:28.063530 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.090340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1334/2692] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:28.090340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.091465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:28.091764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.093221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:28.111832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1359/2693] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:28.114550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.116933 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.190710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=100 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:28.204421 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4a15f-f08c-4223-8499-00f6cbba9f58 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.209721 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:28.210565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:28.210942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:28.211229 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:28.234028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:28.235496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:28.244676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51b90a-25c9-4989-84c9-7f0a0f3a48c7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.249543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1360/2694] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:28.256212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:28.257140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:28.257494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:28.257839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] PATH_INFO: `/v3/projects/e86d133adc4c480d9b4982af8e585e90` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:28.260427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86d133adc4c480d9b4982af8e585e90)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:28.264582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:28.325349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86d133adc4c480d9b4982af8e585e90', 'request_id': 'req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.334954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86d133adc4c480d9b4982af8e585e90', 'request_id': 'req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.337075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.349445 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86d133adc4c480d9b4982af8e585e90', 'request_id': 'req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.352361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7fb48-74ca-4555-b593-946047ed8f71 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.371939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] The token cache is being invalidate because project e86d133adc4c480d9b4982af8e585e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:28.371939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.372453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.377128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1335/2695] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:28 2022] DELETE /identity/v3/projects/e86d133adc4c480d9b4982af8e585e90 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:28.378217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.379666 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.427976 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:28.435652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:28.436884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:28.437952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:28.438650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:28.441964 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.470387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb726b-3e9d-4da4-b30b-fb4c13bf3756 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.477457 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:28.478352 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:28.478625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:28.478909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] PATH_INFO: `/v3/users/e0c63200a4594b23930feec36a01cf9f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:28.485811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c63200a4594b23930feec36a01cf9f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:28.487964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:28.524014 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:28.525898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:28.540230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1361/2696] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:28.563288 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c63200a4594b23930feec36a01cf9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.564368 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c63200a4594b23930feec36a01cf9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.565072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c63200a4594b23930feec36a01cf9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.577569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c63200a4594b23930feec36a01cf9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.594321 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c63200a4594b23930feec36a01cf9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:28.595645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:28.620152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1336/2697] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:28 2022] DELETE /identity/v3/users/e0c63200a4594b23930feec36a01cf9f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:28.916941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.918384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:28.931480 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585988af-31fb-483f-bda5-7be06f9dbcbf service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:28.976785 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:28.978769 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.000669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.001486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.001760 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.002120 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] PATH_INFO: `/v3/users/404cb9c8286d45748ef0ae1371e3973e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.003189 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=404cb9c8286d45748ef0ae1371e3973e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.005135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.063440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '404cb9c8286d45748ef0ae1371e3973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.071677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '404cb9c8286d45748ef0ae1371e3973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.072460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '404cb9c8286d45748ef0ae1371e3973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.075044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380addf0-7bd3-43be-a8fe-c4e8ebd573e6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.079913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.081006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.081303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.081630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] PATH_INFO: `/v3/projects/7c0f2f367daf4a668d83232db352f75b` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.082837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0f2f367daf4a668d83232db352f75b)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.083316 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '404cb9c8286d45748ef0ae1371e3973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.084839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.099799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '404cb9c8286d45748ef0ae1371e3973e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.113928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1362/2698] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:28 2022] DELETE /identity/v3/users/404cb9c8286d45748ef0ae1371e3973e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:29.114660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.116075 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.143975 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:29.144989 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0f2f367daf4a668d83232db352f75b', 'request_id': 'req-7fb14a41-0748-4c62-97dc-406715cc6e51'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.150688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0f2f367daf4a668d83232db352f75b', 'request_id': 'req-7fb14a41-0748-4c62-97dc-406715cc6e51'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.157525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0f2f367daf4a668d83232db352f75b', 'request_id': 'req-7fb14a41-0748-4c62-97dc-406715cc6e51'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.172329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] The token cache is being invalidate because project 7c0f2f367daf4a668d83232db352f75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:29.172640 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.173336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7fb14a41-0748-4c62-97dc-406715cc6e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.174882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1337/2699] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:28 2022] DELETE /identity/v3/projects/7c0f2f367daf4a668d83232db352f75b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:29.176102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606966-7af5-4967-b323-fedc884bc80b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.176420 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606966-7af5-4967-b323-fedc884bc80b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.176733 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f606966-7af5-4967-b323-fedc884bc80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.194444 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-9f606966-7af5-4967-b323-fedc884bc80b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:29.195641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9f606966-7af5-4967-b323-fedc884bc80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.196960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c8deb-81e6-4e46-87d2-3211de1dc9d2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.254147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.255040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.255208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1338/2700] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:29.255420 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.255732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] PATH_INFO: `/v3/users/5b05ad24e50549fc9930b1ded8593a3c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.256992 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b05ad24e50549fc9930b1ded8593a3c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.257128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.257423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.257707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.259122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.277437 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:29.278957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.288812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1339/2701] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:29.290247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.291575 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.306991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:29.314397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b05ad24e50549fc9930b1ded8593a3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.321439 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b05ad24e50549fc9930b1ded8593a3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.322117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b05ad24e50549fc9930b1ded8593a3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.329474 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b05ad24e50549fc9930b1ded8593a3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.342127 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-74bc7321-77d0-4388-a48d-59fa768f1b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b05ad24e50549fc9930b1ded8593a3c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:29.348275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1363/2702] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:29 2022] DELETE /identity/v3/users/5b05ad24e50549fc9930b1ded8593a3c => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:29.349608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b15f3-d32f-46e1-a275-452c7a0bf210 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.350001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b15f3-d32f-46e1-a275-452c7a0bf210 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.350282 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b15f3-d32f-46e1-a275-452c7a0bf210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.359425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7cf3e-9cde-40e8-a550-6eb27b4c3653 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.376599 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-161b15f3-d32f-46e1-a275-452c7a0bf210 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:29.377870 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-161b15f3-d32f-46e1-a275-452c7a0bf210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.414028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.414676 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.414967 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.415248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.417089 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.429758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:29.448313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1364/2703] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:29.449278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.449278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.449278 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.466233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:29.467495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.475224 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1365/2704] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:29.475962 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.482384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.485198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.486584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba979c-0360-47a4-970f-83b442808160 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.491266 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.491918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.492248 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.492529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.493255 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:29.493567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.494872 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1340/2705] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:29.495196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.495691 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.497768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.508658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1366/2706] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:29.512672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.514534 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94170f-87f9-4197-b505-a66d3e91e31d service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.519658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.521045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.521629 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.522134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.523155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0744b0-26e1-43a6-bbe9-fc8dd916ddc8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.530863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.531642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.532094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.532281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.532462 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.533062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.533991 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.535544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.545384 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1341/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:29.566115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1367/2708] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:29.570503 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.580311 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b15e999-1ca8-4b29-9745-5e16edc27754 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.643108 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.643787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.644072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.644380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.645280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.647169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.657730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:29.668822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1342/2709] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:29.673408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.683969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99431c0-f110-4b20-8d65-146f5627b3bc admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.688868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.689532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.689823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.690101 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.690822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.692461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.703593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1368/2710] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:42:29.708680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.735649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665c7ac-59bd-4262-913f-ba74015107ef admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.742713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.744024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.744465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.744908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.746065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.748466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba4b18c-d498-438f-a8ae-36fdf6d4e10e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.763694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1343/2711] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:29.764642 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.778186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09fc461-2cf6-4477-8e0c-ac7c9992ddad admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.784068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.784984 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.785204 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.785570 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] PATH_INFO: `/v3/projects/ba9bcc52b2a145c396d6a954f042d335/users/462ad1b9eb85438c8134028a18ecb6e3/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.806518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9bcc52b2a145c396d6a954f042d335, user_id=462ad1b9eb85438c8134028a18ecb6e3, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.810792 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.836736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1369/2712] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:29 2022] PUT /identity/v3/projects/ba9bcc52b2a145c396d6a954f042d335/users/462ad1b9eb85438c8134028a18ecb6e3/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:29.849723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.849723 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.850006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.867768 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.869634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.903860 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] MFA Rules not processed for user `462ad1b9eb85438c8134028a18ecb6e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:29.905529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:29.955034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63a331-c551-413f-91eb-4860ea456d38 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:29.960638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:29.961699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:29.961848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:29.962056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] PATH_INFO: `/v3/projects/69ec3273b5d640809f9264b3fcd4675e` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:29.963410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ec3273b5d640809f9264b3fcd4675e)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:29.966489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:29.988230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1344/2713] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:29.996673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:29.999239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.014672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16edfaf-1511-4d50-bb45-8a7151180145 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.026825 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ec3273b5d640809f9264b3fcd4675e', 'request_id': 'req-453920d7-00a5-4054-bd19-cbdd90cd684e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.036574 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ec3273b5d640809f9264b3fcd4675e', 'request_id': 'req-453920d7-00a5-4054-bd19-cbdd90cd684e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.042415 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ec3273b5d640809f9264b3fcd4675e', 'request_id': 'req-453920d7-00a5-4054-bd19-cbdd90cd684e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.053828 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] The token cache is being invalidate because project 69ec3273b5d640809f9264b3fcd4675e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:30.054122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.054782 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.059376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1370/2714] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:29 2022] DELETE /identity/v3/projects/69ec3273b5d640809f9264b3fcd4675e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:42:30.059376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.059376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.064147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:30.084592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.085822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.085822 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.086056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] PATH_INFO: `/v3/projects/e31fe158a72c4028b8ebd143c79e09b5` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.087198 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31fe158a72c4028b8ebd143c79e09b5)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.089141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.130144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453920d7-00a5-4054-bd19-cbdd90cd684e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.138405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31fe158a72c4028b8ebd143c79e09b5', 'request_id': 'req-97f78b44-01f7-4724-86d0-329a60a6be03'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.144204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31fe158a72c4028b8ebd143c79e09b5', 'request_id': 'req-97f78b44-01f7-4724-86d0-329a60a6be03'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.150489 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31fe158a72c4028b8ebd143c79e09b5', 'request_id': 'req-97f78b44-01f7-4724-86d0-329a60a6be03'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.167413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] The token cache is being invalidate because project e31fe158a72c4028b8ebd143c79e09b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:30.167744 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.168465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.170098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1345/2715] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:29 2022] DELETE /identity/v3/projects/e31fe158a72c4028b8ebd143c79e09b5 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:30.171053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.172615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.179925 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:30.187793 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e318b1b6386c4a2d9e7f324e8fc2cfe6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.188617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.188997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.189391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.191330 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.250469 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f78b44-01f7-4724-86d0-329a60a6be03 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.255296 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.256269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.256563 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.256847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] PATH_INFO: `/v3/users/5f54d176941a40e2a6affdaeeaea1b9c` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.258163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f54d176941a40e2a6affdaeeaea1b9c)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.260088 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.264004 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.265457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484a19-94bd-46c4-ae8c-9c07fdb0d88d service glance] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.274692 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1371/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:30.275923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d25-a396-4df2-9aa5-ad74be4a0e61 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.276240 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d25-a396-4df2-9aa5-ad74be4a0e61 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.276519 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc490d25-a396-4df2-9aa5-ad74be4a0e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.296216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-fc490d25-a396-4df2-9aa5-ad74be4a0e61 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:30.297491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fc490d25-a396-4df2-9aa5-ad74be4a0e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.306613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1372/2717] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:30.316329 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f54d176941a40e2a6affdaeeaea1b9c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.317100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f54d176941a40e2a6affdaeeaea1b9c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.317675 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f54d176941a40e2a6affdaeeaea1b9c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.318686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.318686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.318686 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.327279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f54d176941a40e2a6affdaeeaea1b9c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.339193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:30.340475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.345499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f54d176941a40e2a6affdaeeaea1b9c'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.351982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1373/2718] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:30.352728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.362397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dda49-0cb4-4272-a4a2-2c345ba761e6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.367485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.367642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1346/2719] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:30 2022] DELETE /identity/v3/users/5f54d176941a40e2a6affdaeeaea1b9c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:30.368468 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.369021 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.369226 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.369862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:30.370168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.372595 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.388156 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1374/2720] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:30.390299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.468506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfe816-c4a3-4233-96ed-423fdc4ba377 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.473373 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.474504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.474504 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.474730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.475484 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.477507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.509330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1347/2721] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:30.512851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.523429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505da21-9f22-49d6-a0dc-66b29daab464 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.582496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:30.598089 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.598887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.599150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.599435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.600404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.602764 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.633242 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1375/2722] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/users => generated 283 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:30.635039 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.646885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d5f9d-cff7-4f9d-80cf-f33f6328b326 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.652212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.652889 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.653618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.653618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.654307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.656572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.659106 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:30.684390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1348/2723] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:30.687864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.688040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.695206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.699674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85e9ef-3004-4f36-b7d8-9e4685b79549 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.712859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b75a-bff1-47a8-94c9-e4589ad3b541 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.716396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.717540 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.718673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.719277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.719906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.720270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.720671 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.721487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] PATH_INFO: `/v3/projects/1da724350bb646d4b80c8536079213c8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.722291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.722467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da724350bb646d4b80c8536079213c8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.725182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.726607 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.764749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1376/2724] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:30.765823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:30.780752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e89177-4ec0-49a2-90d7-6f436c92cc10 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:30.782099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da724350bb646d4b80c8536079213c8', 'request_id': 'req-5f19ca71-5231-4175-b132-4f993ae5889f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.788597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da724350bb646d4b80c8536079213c8', 'request_id': 'req-5f19ca71-5231-4175-b132-4f993ae5889f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.797473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da724350bb646d4b80c8536079213c8', 'request_id': 'req-5f19ca71-5231-4175-b132-4f993ae5889f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.814490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] The token cache is being invalidate because project 1da724350bb646d4b80c8536079213c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:30.814840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.815742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f19ca71-5231-4175-b132-4f993ae5889f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:30.820159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1349/2725] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:30 2022] DELETE /identity/v3/projects/1da724350bb646d4b80c8536079213c8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:30.822742 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91758-5e61-49ce-beda-31c83277a74d None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.823022 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91758-5e61-49ce-beda-31c83277a74d None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.823359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b91758-5e61-49ce-beda-31c83277a74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.841972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-53b91758-5e61-49ce-beda-31c83277a74d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:30.843195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-53b91758-5e61-49ce-beda-31c83277a74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.856704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:30.857676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.857950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.858233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] PATH_INFO: `/v3/projects/5edcf34bc63a4954a93eb2b4c72869dd/users/e21c4b08ff064b228dd55398d14fed5a/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.874951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5edcf34bc63a4954a93eb2b4c72869dd, user_id=e21c4b08ff064b228dd55398d14fed5a, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:30.877672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09fa61-9b76-42d1-8130-c2f9af5e201e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:30.900675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1377/2726] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:30 2022] PUT /identity/v3/projects/5edcf34bc63a4954a93eb2b4c72869dd/users/e21c4b08ff064b228dd55398d14fed5a/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:30.902203 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.902494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.902777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.920244 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1350/2727] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:30.925543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:30.925843 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:30.926128 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:30.941888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] MFA Rules not processed for user `e21c4b08ff064b228dd55398d14fed5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:30.943113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:30.946681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:30.947951 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.010031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1351/2728] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:31.011350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1378/2729] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:31.014454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.022304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.023649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.023777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868995c-caad-4201-8964-5daf4d5d28c1 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.028882 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.029535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.029875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.030158 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.030888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:31.031193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.032839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.047753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1352/2730] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:31.049431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.059091 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e9eeff-4b66-4572-be2b-f1a1f08a5a67 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.063645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.064303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.064585 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.064858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.065541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.066093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:31.067405 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.071934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:31.083950 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeafa284-6548-45b4-bdc1-5bb7704690d2 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.088516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.089288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.090083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.090361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.095859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1353/2731] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:31.099532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.099677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.101267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.110953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1379/2732] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:31.162993 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c7f0-6213-4617-bf24-f469fbdf0d8d admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.168210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.168840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.169152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.169421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.170097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.171982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.191608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1354/2733] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:31 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:42:31.195807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.207335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43c55-c4db-43ed-92e1-ffa617f461dc service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.212168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.212823 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.213110 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.213391 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.214112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.216338 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.226515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1380/2734] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:31.231784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.243319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c9ad5-8cff-4fb0-8b97-eba1166183fb admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.248490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.249426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.249791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.249988 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.250858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.252726 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1d63f-bfba-44c5-af02-69b6f53bbddf admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.264878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1355/2735] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:31.269654 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.283152 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61f500-47ec-4f52-b597-83726cd34ad3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.287816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.288871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.289158 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.289440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] PATH_INFO: `/v3/projects/90c1c3b0c20946b0b313b879794f63dc/users/886bff33c3fb4c17918de520ea8ce804/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.309500 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c1c3b0c20946b0b313b879794f63dc, user_id=886bff33c3fb4c17918de520ea8ce804, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.312193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.329494 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1381/2736] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:31 2022] PUT /identity/v3/projects/90c1c3b0c20946b0b313b879794f63dc/users/886bff33c3fb4c17918de520ea8ce804/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:31.341513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.341803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.342700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.377734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] MFA Rules not processed for user `886bff33c3fb4c17918de520ea8ce804`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:31.379228 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.461143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1356/2737] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:31.474168 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.532386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6c6c4-e5ab-4a49-be25-d68ac1375c0a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.536041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.536699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.536982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.537259 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.545449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.546857 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3d6a6-2431-451f-9131-930632d6d3f1 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.555378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1382/2738] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:31.575987 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.636890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:31.643864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf1fff-83cd-4e33-a80f-81bbe1c97616 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:31.651625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:31.653359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.653828 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.654248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] PATH_INFO: `/v3/users/886bff33c3fb4c17918de520ea8ce804` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.655800 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886bff33c3fb4c17918de520ea8ce804)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:31.658753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:31.745249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886bff33c3fb4c17918de520ea8ce804'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:31.750641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886bff33c3fb4c17918de520ea8ce804'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:31.752529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886bff33c3fb4c17918de520ea8ce804'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:31.766206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886bff33c3fb4c17918de520ea8ce804'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:31.786941 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886bff33c3fb4c17918de520ea8ce804'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:31.801295 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1357/2739] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:31 2022] DELETE /identity/v3/users/886bff33c3fb4c17918de520ea8ce804 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:31.805901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ccc54-64da-4ec3-9609-0d2f494bcc95 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.806190 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ccc54-64da-4ec3-9609-0d2f494bcc95 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.806546 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ccc54-64da-4ec3-9609-0d2f494bcc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.826794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ae1ccc54-64da-4ec3-9609-0d2f494bcc95 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:31.828146 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ae1ccc54-64da-4ec3-9609-0d2f494bcc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.907839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1383/2740] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:31.918448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:31.920210 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.972502 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:31.972771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:31.973231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:31.991649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:31.992915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:31.998133 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5e09d-be04-47b2-9ab7-1ffdb22e8a04 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:32.003359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:32.004038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:32.004689 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:32.004962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:32.020746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:32.022735 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:32.050091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:32.051079 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1358/2741] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:32.079278 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:32.081164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1384/2742] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:42:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:32.087853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:32.196961 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cf9ab-534f-4cd4-97d9-cb7a73a628aa service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:32.203538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:32.204343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:32.204706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:32.205057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:32.219749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:32.221418 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7262a04-70b2-411b-8765-a1ed64d3a8c2 service swift] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:32.232216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1359/2743] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:42:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 07:42:32.455906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:32.458155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:32.501752 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e18a673-38c9-48a2-acc8-f85e9d982136 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:32.513469 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:32.514621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:32.516481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:32.516481 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] PATH_INFO: `/v3/projects/90c1c3b0c20946b0b313b879794f63dc` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:32.521315 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c1c3b0c20946b0b313b879794f63dc)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:32.528748 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:32.676068 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c1c3b0c20946b0b313b879794f63dc', 'request_id': 'req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:32.689107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c1c3b0c20946b0b313b879794f63dc', 'request_id': 'req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:32.708532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c1c3b0c20946b0b313b879794f63dc', 'request_id': 'req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:32.740262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] The token cache is being invalidate because project 90c1c3b0c20946b0b313b879794f63dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:32.740571 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:32.746496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6a2670f1-aa92-420e-b62c-2f0e2bd6d4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:32.749829 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1385/2744] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:32 2022] DELETE /identity/v3/projects/90c1c3b0c20946b0b313b879794f63dc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:32.754677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:32.755302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:32.755302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:32.784981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:32.798231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:32.799578 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:32.972478 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1360/2745] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:32.980147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:32.981150 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:32.981861 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.014475 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:33.016006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:33.037452 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1386/2746] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:33.041507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.059931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1676052a-583c-47ec-ad74-b6f778ba3b4f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.067520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.068622 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.069185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.069687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.072315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:33.073170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.076446 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.101454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1361/2747] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:33.105334 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.122960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d417a-9b2e-4ca5-85f6-154fffa74402 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.140867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.141554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.141835 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.142123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.142836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.144778 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.236381 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1387/2748] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:33 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:33.242430 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.255968 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3c4ca-329a-4809-ab17-ba363dee02ff admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.362706 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.363403 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.363687 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.363970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.364730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.368219 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.423346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1362/2749] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:33 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:33.424668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.440960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5be0e1-31ae-424c-82ef-a0682fff3e7b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.457351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.458054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.458304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.458705 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.459460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.461289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.490082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1388/2750] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:33.491995 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.506727 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c1c19-ce41-4b07-af29-258d484fb489 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.556627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.556627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.556627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.556627 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.558615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.558615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380156d1-e81d-42c8-9cf8-5a830b359a43 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.585535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1363/2751] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:33.589746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:33.604704 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34855214-3e26-48c1-ae5d-9dbf626a73e9 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:33.611291 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:33.613615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.613912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.614250 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] PATH_INFO: `/v3/projects/d9adf0157744463b99e339f6e944c01b/users/06f88014dd184960bae58faad4f7962a/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.657042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9adf0157744463b99e339f6e944c01b, user_id=06f88014dd184960bae58faad4f7962a, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:33.660031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:33.679489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1389/2752] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:33 2022] PUT /identity/v3/projects/d9adf0157744463b99e339f6e944c01b/users/06f88014dd184960bae58faad4f7962a/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:33.691086 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:33.691505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:33.691890 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:33.752945 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] MFA Rules not processed for user `06f88014dd184960bae58faad4f7962a`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:33.756260 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:34.026269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1364/2753] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:34.121028 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:34.122863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:34.272541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b707ca9-cef0-424f-9462-c2f1d2a84161 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:34.283125 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:34.283941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:34.287549 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:34.288276 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:34.319300 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:34.320969 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37292ab7-ae83-4cd6-9df0-fb788c8241ec service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:34.346225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1390/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:35.784740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:35.786570 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:35.795098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:35.865875 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d146c6-b395-4e2c-a97d-91899dd79a58 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:35.875672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:35.876266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:35.876674 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:35.877042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] PATH_INFO: `/v3/users/a18382e32fd5486e8ef7fd0d48979c3a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:35.878525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18382e32fd5486e8ef7fd0d48979c3a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:35.880864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:35.942250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18382e32fd5486e8ef7fd0d48979c3a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:35.942918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18382e32fd5486e8ef7fd0d48979c3a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:35.943546 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18382e32fd5486e8ef7fd0d48979c3a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:35.953536 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18382e32fd5486e8ef7fd0d48979c3a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:35.970118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18382e32fd5486e8ef7fd0d48979c3a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:35.985156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1365/2755] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:35 2022] DELETE /identity/v3/users/a18382e32fd5486e8ef7fd0d48979c3a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:35.989589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:35.990054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:35.990215 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:36.014536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:36.015775 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:36.038145 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:36.168100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1391/2756] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:36.181122 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:36.189766 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:36.385270 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124df185-bc05-423b-897e-da9d43694e6b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:36.389643 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:36.390281 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:36.390571 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:36.391032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:36.405419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:36.407019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:36.429248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1366/2757] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:36.907970 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:36.909888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:36.937983 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd90a55-099d-4ae8-a700-97a50bf8d6fc None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:36.943662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:36.944947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:36.945305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:36.945762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] PATH_INFO: `/v3/projects/9bbf2f328485486eba70f5e50e564897` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:36.947078 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bbf2f328485486eba70f5e50e564897)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:36.949899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:37.013580 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bbf2f328485486eba70f5e50e564897', 'request_id': 'req-87a3447a-ced0-498a-bf14-42c6b3e23638'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.027918 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bbf2f328485486eba70f5e50e564897', 'request_id': 'req-87a3447a-ced0-498a-bf14-42c6b3e23638'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.034883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bbf2f328485486eba70f5e50e564897', 'request_id': 'req-87a3447a-ced0-498a-bf14-42c6b3e23638'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.047223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] The token cache is being invalidate because project 9bbf2f328485486eba70f5e50e564897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:37.047529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.048232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.053164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1392/2758] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:36 2022] DELETE /identity/v3/projects/9bbf2f328485486eba70f5e50e564897 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:37.054268 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:37.070156 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e401c0f-4f5b-4e45-8f70-baabca1aadef service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:37.097256 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:37.135510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:37.138019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:37.138019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:37.138871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] PATH_INFO: `/v3/users/6aabbcbbfade40d996db9081ba41264d` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:37.140147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6aabbcbbfade40d996db9081ba41264d)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:37.142520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:37.201717 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6aabbcbbfade40d996db9081ba41264d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.202392 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6aabbcbbfade40d996db9081ba41264d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.203017 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6aabbcbbfade40d996db9081ba41264d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.210814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6aabbcbbfade40d996db9081ba41264d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.239874 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-088e2af9-4ba9-41af-a6f3-20c0bbaa6c9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6aabbcbbfade40d996db9081ba41264d'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.276292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1367/2759] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:37 2022] DELETE /identity/v3/users/6aabbcbbfade40d996db9081ba41264d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:37.612736 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:37.614262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:37.636387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3447a-ced0-498a-bf14-42c6b3e23638 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:37.705403 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:37.706396 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:37.706759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:37.707041 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] PATH_INFO: `/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:37.708364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfef34a3e9b44df683d8baaefdbf81c1)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:37.710510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:37.768898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:37.777923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfef34a3e9b44df683d8baaefdbf81c1', 'request_id': 'req-48bc7c30-35db-418b-93ee-bb6d7576c756'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.789813 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfef34a3e9b44df683d8baaefdbf81c1', 'request_id': 'req-48bc7c30-35db-418b-93ee-bb6d7576c756'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.798520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfef34a3e9b44df683d8baaefdbf81c1', 'request_id': 'req-48bc7c30-35db-418b-93ee-bb6d7576c756'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.810938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] The token cache is being invalidate because project dfef34a3e9b44df683d8baaefdbf81c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:37.811326 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.812054 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-48bc7c30-35db-418b-93ee-bb6d7576c756 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:37.813696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1393/2760] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:37 2022] DELETE /identity/v3/projects/dfef34a3e9b44df683d8baaefdbf81c1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:37.818713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:37.819040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:37.819353 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:37.838340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:37.840232 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:37.914610 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1368/2761] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:37.975360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:37.975668 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:37.975944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:37.996362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:37.997604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:38.017862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1394/2762] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:38.021768 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.032075 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f741f6-b183-4d09-b128-9650d0980534 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.036863 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.037525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.037880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.038083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.038821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:38.039135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.040838 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.058367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1369/2763] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:38.061366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.073909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15480c-39ba-4a47-ac1a-83c152ad45d0 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.082428 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.083401 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.083698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.083982 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.084796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.086719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.140186 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1395/2764] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:38.142407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.153161 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fb4de-8612-4c2f-930f-6d7a5138b928 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.187684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:38.224157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.224852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.225138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.225423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.226126 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.228087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.250496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1370/2765] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:38.254529 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.266389 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac706d09-233a-45a5-a0a5-e3ce334083a5 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.271388 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.272071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.272433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.272719 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.273460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.275073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.295634 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1396/2766] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:38.296471 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.310561 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cbb355-88d9-4314-a10c-7250b55f26f0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.317151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.317992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.318348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.318763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.319639 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.321447 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f3f00-019f-4ab5-8231-8d6d257313b0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.333113 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1371/2767] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:42:38.337290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.348584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ac759-4d43-4f07-a06c-933bb8fce210 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.353367 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.354356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.354644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.354956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] PATH_INFO: `/v3/projects/36a46c68fed647d0811d7eedde4baf09/users/d8754c7c68e44d1c8ac00b6e52849a9f/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.373567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a46c68fed647d0811d7eedde4baf09, user_id=d8754c7c68e44d1c8ac00b6e52849a9f, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.380915 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.401202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1397/2768] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:38 2022] PUT /identity/v3/projects/36a46c68fed647d0811d7eedde4baf09/users/d8754c7c68e44d1c8ac00b6e52849a9f/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:38.412656 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.412950 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.413226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.442099 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] MFA Rules not processed for user `d8754c7c68e44d1c8ac00b6e52849a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:38.443410 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:38.511722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1372/2769] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:38.516288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.586785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682ba7b7-39e7-4806-b6e6-85114c694225 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.591347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.592001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.592307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.592583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.593265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.595180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.621956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1398/2770] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:38.626369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.637876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f22ddf-c6e7-4ff2-87bd-ab921d282ecd None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.693777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.694473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.694736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.695020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.695714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.697664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.720669 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1373/2771] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:38.724340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.734516 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688260-3a39-42ce-892b-4b6d146de156 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.739526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.740225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.740513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.740791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.741534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.743182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.754603 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1399/2772] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:38.758677 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.769603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b71bf8-a64b-4465-9fb7-4570a778ea25 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.774314 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.775189 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.775473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.775754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] PATH_INFO: `/v3/projects/4b42f30f133a4f32b01f3c4a122299e9/users/26157d5169dc479f8ca0fd63ee49a504/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.790369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b42f30f133a4f32b01f3c4a122299e9, user_id=26157d5169dc479f8ca0fd63ee49a504, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.793057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.805747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1374/2773] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:38 2022] PUT /identity/v3/projects/4b42f30f133a4f32b01f3c4a122299e9/users/26157d5169dc479f8ca0fd63ee49a504/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:38.810419 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.820677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9fa54-d0b1-4312-bb52-b27ee226789a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.859901 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:38.881395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.882071 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.882358 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.882637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.883372 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.885011 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a7b7b-61d3-4746-9ad0-110884faee8a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.895625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1400/2774] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:38.899641 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:38.906327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:38.910083 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70dc280-4244-42db-a837-a72255737f3a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:38.915741 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:38.916665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.916948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.917737 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] PATH_INFO: `/v3/projects/4b42f30f133a4f32b01f3c4a122299e9/users/26157d5169dc479f8ca0fd63ee49a504/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.919931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b42f30f133a4f32b01f3c4a122299e9, user_id=26157d5169dc479f8ca0fd63ee49a504, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:38.922607 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:38.934623 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1375/2775] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:38 2022] PUT /identity/v3/projects/4b42f30f133a4f32b01f3c4a122299e9/users/26157d5169dc479f8ca0fd63ee49a504/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:38.945053 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:38.945392 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:38.945762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:38.971747 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] MFA Rules not processed for user `26157d5169dc479f8ca0fd63ee49a504`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:38.973082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:39.041460 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1401/2776] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:39.053582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:39.055302 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:39.114709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f3ea8a-4deb-4653-b00f-cb68549ad267 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:39.118308 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:39.118949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:39.119237 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:39.119515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:39.130138 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:39.131814 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:39.141964 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1376/2777] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:39.252147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:39.261967 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d30b77c-d029-401a-9357-5716776d0a02 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:39.265758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:39.266517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:39.266875 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:39.267543 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:39.334322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:39.335759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f6157-4e61-48b9-927f-ae9e712f851c service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:39.345335 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1402/2778] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:44.151224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:44.223956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec477b-9cf1-4d67-a061-f8593c34bf8f service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:44.228525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:44.229391 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:44.230248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:44.230473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] PATH_INFO: `/v3/users/d8754c7c68e44d1c8ac00b6e52849a9f` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:44.231662 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8754c7c68e44d1c8ac00b6e52849a9f)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:44.233595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:44.280544 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8754c7c68e44d1c8ac00b6e52849a9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.281426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8754c7c68e44d1c8ac00b6e52849a9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.282111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8754c7c68e44d1c8ac00b6e52849a9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.291603 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8754c7c68e44d1c8ac00b6e52849a9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.304793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8754c7c68e44d1c8ac00b6e52849a9f'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.318486 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1377/2779] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:44 2022] DELETE /identity/v3/users/d8754c7c68e44d1c8ac00b6e52849a9f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:44.323896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:44.324414 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:44.324615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:44.351395 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:44.353239 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:44.422398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=100 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:44.434746 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1403/2780] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:44.445877 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:44.447220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:44.472181 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:44.525487 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992ef28-eeff-4d18-9db7-1ab71d8dc57e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:44.529864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:44.530502 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:44.530788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:44.531063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:44.553693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:44.553693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:44.564566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1378/2781] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:44.849785 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:44.851343 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:44.866698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedd64-f630-45ac-b0aa-2c4100c4bca5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:44.877121 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:44.878017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:44.878302 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:44.878568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] PATH_INFO: `/v3/projects/36a46c68fed647d0811d7eedde4baf09` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:44.879757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a46c68fed647d0811d7eedde4baf09)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:44.881741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:44.927098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a46c68fed647d0811d7eedde4baf09', 'request_id': 'req-fb7e436d-a744-4bd2-b213-23a972df2c39'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.933505 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a46c68fed647d0811d7eedde4baf09', 'request_id': 'req-fb7e436d-a744-4bd2-b213-23a972df2c39'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.939100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a46c68fed647d0811d7eedde4baf09', 'request_id': 'req-fb7e436d-a744-4bd2-b213-23a972df2c39'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.950613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] The token cache is being invalidate because project 36a46c68fed647d0811d7eedde4baf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:44.950931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.951560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:44.953111 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1404/2782] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:44 2022] DELETE /identity/v3/projects/36a46c68fed647d0811d7eedde4baf09 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:44.957025 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:44.967038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076c599-922a-4bce-ad16-caa22d866238 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.021565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.022425 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.022713 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.022992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] PATH_INFO: `/v3/users/26157d5169dc479f8ca0fd63ee49a504` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.024263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26157d5169dc479f8ca0fd63ee49a504)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.026171 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.070728 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26157d5169dc479f8ca0fd63ee49a504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.071381 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26157d5169dc479f8ca0fd63ee49a504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.071992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26157d5169dc479f8ca0fd63ee49a504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.080312 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26157d5169dc479f8ca0fd63ee49a504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.093301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b047fa8c-42ea-4a06-badb-03270b853c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26157d5169dc479f8ca0fd63ee49a504'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.105747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1379/2783] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:44 2022] DELETE /identity/v3/users/26157d5169dc479f8ca0fd63ee49a504 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:45.417019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.418593 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:45.434791 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e436d-a744-4bd2-b213-23a972df2c39 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.492745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.493708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.493986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.494263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] PATH_INFO: `/v3/projects/4b42f30f133a4f32b01f3c4a122299e9` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.495948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b42f30f133a4f32b01f3c4a122299e9)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.497866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.544361 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b42f30f133a4f32b01f3c4a122299e9', 'request_id': 'req-ba8725af-a888-49f4-8db4-70267186b53e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.553137 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b42f30f133a4f32b01f3c4a122299e9', 'request_id': 'req-ba8725af-a888-49f4-8db4-70267186b53e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.559490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b42f30f133a4f32b01f3c4a122299e9', 'request_id': 'req-ba8725af-a888-49f4-8db4-70267186b53e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.570751 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] The token cache is being invalidate because project 4b42f30f133a4f32b01f3c4a122299e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:45.571043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.571697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ba8725af-a888-49f4-8db4-70267186b53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:45.573895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1405/2784] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:45 2022] DELETE /identity/v3/projects/4b42f30f133a4f32b01f3c4a122299e9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:45.579045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.579495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.579651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.599217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:45.600789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:45.632758 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:45.662913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1380/2785] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:45.672765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.673058 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.673344 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.690907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:45.702251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:45.703635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:45.712712 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1406/2786] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:45.717235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.726787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d891e4-c986-455f-8b36-e952966c7284 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.731788 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.732477 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.732754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.733030 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.733753 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:45.734056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.735660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.747819 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1381/2787] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:45.752210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.761210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a4cfe6-fcfa-4c2e-a612-f46122605fa5 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.765808 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.766446 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.766726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.766999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.767687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.769591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.795615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1407/2788] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:45 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:45.799892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.817494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631aff9-91cf-4619-bf9b-6e5cc1ffa1fd admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.874013 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.874871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.875154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.875432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.876149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.878049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.896866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1382/2789] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:45 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:45.900841 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.910911 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1ca96-b7c3-4261-b786-20c41615d3a2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.915410 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.916050 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.916360 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.916638 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.917867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.919440 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.929558 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1408/2790] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:45.933754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.943495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc8a20-bc5a-468e-a8a4-8904490efce7 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.948000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.948725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.949009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.949285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.950000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.951587 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af6f56-4491-4454-9e53-de1c002ec8ca admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:45.961490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1383/2791] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:42:45.965340 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:45.974920 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29721b4d-a1f9-4da9-85d6-127436a8fb9a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:45.979986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:45.980872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:45.981154 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:45.981426 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] PATH_INFO: `/v3/projects/951f595309254a74b6abdd623d5c4e36/users/b2d9a24ac7424991b4cbd775265de4d8/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:45.995957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=951f595309254a74b6abdd623d5c4e36, user_id=b2d9a24ac7424991b4cbd775265de4d8, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:45.998594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:46.011100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1409/2792] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:45 2022] PUT /identity/v3/projects/951f595309254a74b6abdd623d5c4e36/users/b2d9a24ac7424991b4cbd775265de4d8/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:46.021024 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.021315 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.021595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.049835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] MFA Rules not processed for user `b2d9a24ac7424991b4cbd775265de4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:46.051100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:46.114515 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1384/2793] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:46 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:46.144434 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:46.146109 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:46.201954 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61a66d-74b8-4fd0-92ca-7d1bc04cf8b3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:46.205584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:46.206237 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.206550 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.206833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.216196 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:46.217620 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f07de-26d4-461b-b338-e3c99a71f459 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:46.226681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1410/2794] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:46.241415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:46.274771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:46.305235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f9e0a-f7e9-4c85-a015-d4ae2da4b941 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:46.309756 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:46.310626 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.310908 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.311183 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] PATH_INFO: `/v3/users/b2d9a24ac7424991b4cbd775265de4d8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.312532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d9a24ac7424991b4cbd775265de4d8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:46.314442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:46.361057 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d9a24ac7424991b4cbd775265de4d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.361720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d9a24ac7424991b4cbd775265de4d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.362367 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d9a24ac7424991b4cbd775265de4d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.369358 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d9a24ac7424991b4cbd775265de4d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.381647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d9a24ac7424991b4cbd775265de4d8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.394085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1385/2795] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:42:46 2022] DELETE /identity/v3/users/b2d9a24ac7424991b4cbd775265de4d8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:46.398588 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.398877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.399155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.416496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:46.417738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:46.475671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1411/2796] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:42:46.485129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:46.486506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:46.551432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75512a1e-ede7-46fa-b54c-3ae00a0b2197 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:46.555199 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:46.555868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.556186 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.556472 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.565506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:46.566934 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda4319-5e0b-4df6-9ddd-13fae5a5a36e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:46.576853 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1386/2797] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:46.844720 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:46.846685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:46.870613 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87212736-a69c-46ff-8a4d-524ed51e5ac6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:46.875895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:46.876826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.877153 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.877378 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] PATH_INFO: `/v3/projects/951f595309254a74b6abdd623d5c4e36` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.878590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=951f595309254a74b6abdd623d5c4e36)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:46.880531 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:46.920842 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:46.928698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '951f595309254a74b6abdd623d5c4e36', 'request_id': 'req-6df38ea6-8476-45b7-935d-ffed16a8b07c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.936617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '951f595309254a74b6abdd623d5c4e36', 'request_id': 'req-6df38ea6-8476-45b7-935d-ffed16a8b07c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.941989 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '951f595309254a74b6abdd623d5c4e36', 'request_id': 'req-6df38ea6-8476-45b7-935d-ffed16a8b07c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.953674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] The token cache is being invalidate because project 951f595309254a74b6abdd623d5c4e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:42:46.953957 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.954623 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6df38ea6-8476-45b7-935d-ffed16a8b07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:42:46.956265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1412/2798] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:42:46 2022] DELETE /identity/v3/projects/951f595309254a74b6abdd623d5c4e36 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:46.961073 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:46.961524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:46.961638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:46.978830 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:46.979983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.020354 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:47.043377 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1387/2799] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:47.051106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.051400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.051681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.070665 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:47.071921 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.080732 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1413/2800] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:47.085107 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.095142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d3d7d-e85d-4ec0-b57e-c0caa828304c None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.100795 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.101458 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.101743 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.102021 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.102749 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:42:47.103060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.105211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.120672 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1388/2801] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:42:47.122435 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.132271 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d3363c-da32-4ab4-8f3a-94b40a181f9d None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.137598 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.138512 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.141382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.141826 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.142557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.144514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.179917 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1414/2802] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:42:47 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:47.181326 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.192239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f59ed-5036-47d2-9ea8-e00a03eb5e80 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.265867 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.266581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.266864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.267140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.267880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.269813 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.297858 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1389/2803] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:42:47 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:42:47.302148 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.313552 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025fd12-932b-4605-94f0-669378b856d1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.319479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.320398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.320864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.321470 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.322619 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.332373 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.349244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1415/2804] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:47.350642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.365839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae3b2c-a3ae-44ea-a26d-ba0bdfd58a3f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.369493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.370008 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.370832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.371111 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.371835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.373529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef6b301-6dce-4657-b226-dd626abfa040 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.387245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1390/2805] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:42:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:42:47.388444 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.398876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163d978-e63e-49ca-8c3c-626796187838 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.404184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.405567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.405926 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.406398 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] PATH_INFO: `/v3/projects/044e8882fac7469a95707bc6ff31dab4/users/e2b9adbe60b340f4a7e3ef4e55ba5104/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.429269 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=044e8882fac7469a95707bc6ff31dab4, user_id=e2b9adbe60b340f4a7e3ef4e55ba5104, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.433485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.455025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1416/2806] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:42:47 2022] PUT /identity/v3/projects/044e8882fac7469a95707bc6ff31dab4/users/e2b9adbe60b340f4a7e3ef4e55ba5104/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:42:47.465817 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.466117 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.466402 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.487682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.489514 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.498543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] MFA Rules not processed for user `e2b9adbe60b340f4a7e3ef4e55ba5104`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:42:47.500137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.570356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8fa35-2da6-4bf9-93bf-7a17f16a1398 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.575759 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.576547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.576988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.577383 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.577652 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1391/2807] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:42:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:42:47.579780 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.600620 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:47.602346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:47.682532 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:47.682922 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:42:47.710327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78d889-42d5-48c7-b0fc-3b9579fe36a0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:47.715359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.716628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:47.717541 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:47.717887 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.718242 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:47.718720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:47.730255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:47.732552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:47.742816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1392/2808] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:42:47.748928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1417/2809] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:50.676431 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:42:50.677834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:50.694915 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54ee4e-c692-4a24-8b4f-94cfec906e57 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:42:50.700557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:42:50.701206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:42:50.701513 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:42:50.701787 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:42:50.704372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:42:50.718531 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:42:50.720164 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf33ca-a646-4665-b2e9-fe87107f412c service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:42:50.731772 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1393/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:42:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:42:57.744951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=10 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:00.726348 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=14 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:03.973594 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:03.975142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.050805 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23266fa-c225-4b83-b927-91262fd2e6a5 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:04.056017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:04.056938 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.057216 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.057498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] PATH_INFO: `/v3/users/462ad1b9eb85438c8134028a18ecb6e3` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.059281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462ad1b9eb85438c8134028a18ecb6e3)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:04.061310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:04.127621 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462ad1b9eb85438c8134028a18ecb6e3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.135522 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462ad1b9eb85438c8134028a18ecb6e3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.137073 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462ad1b9eb85438c8134028a18ecb6e3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.145844 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462ad1b9eb85438c8134028a18ecb6e3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.167279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462ad1b9eb85438c8134028a18ecb6e3'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.174080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1418/2811] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:03 2022] DELETE /identity/v3/users/462ad1b9eb85438c8134028a18ecb6e3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:04.178698 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.179215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.179376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.197163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:04.198363 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.267223 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1394/2812] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:04.277691 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:04.279178 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.345789 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71556af-c7e5-4284-b40a-ad3a9ca75cb8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:04.349626 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:04.350274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.350560 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.350843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.360847 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:04.362313 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2275666-00fb-485f-9365-c29e168cdcec service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:04.374637 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1419/2813] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:04.713172 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:04.714840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.734963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2a7e4-561b-4755-b91a-684164dcb478 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:04.752588 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:04.753600 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.753957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.754310 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] PATH_INFO: `/v3/projects/ba9bcc52b2a145c396d6a954f042d335` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.755665 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9bcc52b2a145c396d6a954f042d335)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:04.757971 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:04.814559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9bcc52b2a145c396d6a954f042d335', 'request_id': 'req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.820547 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9bcc52b2a145c396d6a954f042d335', 'request_id': 'req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.829141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9bcc52b2a145c396d6a954f042d335', 'request_id': 'req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.843625 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] The token cache is being invalidate because project ba9bcc52b2a145c396d6a954f042d335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:04.843916 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.844566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:04.846100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1395/2814] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:04 2022] DELETE /identity/v3/projects/ba9bcc52b2a145c396d6a954f042d335 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:04.851498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce19-45b9-457d-8d37-fff953d89975 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.851867 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce19-45b9-457d-8d37-fff953d89975 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.852244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebce19-45b9-457d-8d37-fff953d89975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.877708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-62ebce19-45b9-457d-8d37-fff953d89975 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:04.878941 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-62ebce19-45b9-457d-8d37-fff953d89975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.890337 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:04.891763 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.900288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:04.956433 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1420/2815] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:04.964022 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.964382 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.964669 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.968601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07be8e-d452-4bb0-b7ac-fc4e58abf3b8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:04.974617 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:04.975427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:04.975748 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:04.976032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] PATH_INFO: `/v3/users/06f88014dd184960bae58faad4f7962a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:04.977464 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f88014dd184960bae58faad4f7962a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:04.979356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:04.982898 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:04.984084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:04.997342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1421/2816] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:05.001688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.019168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:05.019678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77468f-6362-4285-88e7-06878a7cbd23 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.025673 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.026615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.027103 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.027586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.028399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:43:05.028678 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.030307 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.042844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f88014dd184960bae58faad4f7962a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.043668 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f88014dd184960bae58faad4f7962a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.044330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f88014dd184960bae58faad4f7962a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.050123 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1422/2817] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:43:05.054763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.055747 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f88014dd184960bae58faad4f7962a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.056724 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.072738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e440394-a06d-45de-80b7-262365887398 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.074395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-b9a4ff61-2825-4f76-bb75-d34c66f34c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f88014dd184960bae58faad4f7962a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.077839 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.078601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.079049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.079496 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.080422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.082406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.092576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1396/2818] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:04 2022] DELETE /identity/v3/users/06f88014dd184960bae58faad4f7962a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:05.095257 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.095554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.095837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.114688 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:05.115202 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1423/2819] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:05.116035 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.119337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.186473 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1397/2820] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:05.187006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a6735-f10c-4679-83c4-54e68575c5f6 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.191947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.192738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.193040 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.193305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.194009 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.196134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.198568 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.200848 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.221170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1424/2821] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:05.225615 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.241234 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5052d19-53c1-4e0d-8672-7326ec83c77e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.247563 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.248461 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.249019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.249509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.250076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.251698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.262979 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1425/2822] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:43:05.266415 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d469d-5803-4338-bd23-4c1a29ec6f91 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.267482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.268939 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.269598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.269932 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.270168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.278244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7deae-d06d-4ad7-ae68-b91ef50ade54 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.281682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.283193 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.283837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.284526 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.284814 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.285091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.285816 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.287832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4d01e8-18d8-4dfe-8c20-8a911b05d48a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.294185 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1398/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:05.303776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1426/2824] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:43:05.305211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.315960 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3600394f-a123-4ae6-8bfe-b6df11fbf09e service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.320693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.321576 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.321871 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.322154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] PATH_INFO: `/v3/projects/447e2de61cd84ac3887260319161e817/users/fcf7034654e24046be8a82cb140393bd/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.338182 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=447e2de61cd84ac3887260319161e817, user_id=fcf7034654e24046be8a82cb140393bd, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.341031 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.356601 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1399/2825] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:05 2022] PUT /identity/v3/projects/447e2de61cd84ac3887260319161e817/users/fcf7034654e24046be8a82cb140393bd/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:05.366497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.366806 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.367042 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.398172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] MFA Rules not processed for user `fcf7034654e24046be8a82cb140393bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:05.399399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.439790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:05.479707 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1427/2826] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:05.481479 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.509725 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:05.557685 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f8bbc-409e-4461-b903-540960885a4b admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.564299 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.564983 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.565419 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.565705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.566565 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.568793 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.598925 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.600959 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.612659 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1400/2827] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:05.614047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.616082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340eb63c-d0bf-4693-8712-33ab3ae5143a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.682491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.683285 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f4cec-9c2f-4dcc-803e-1e203804b98a admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.683429 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.683802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.684001 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] PATH_INFO: `/v3/projects/d9adf0157744463b99e339f6e944c01b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.685205 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9adf0157744463b99e339f6e944c01b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.687129 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.688246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.688982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.689213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.689491 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.690197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.692187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.716144 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1401/2828] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:05.719923 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.729876 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5ecfb-b7d4-4e05-8396-1cca30cd9197 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.735375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.736056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.736372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.736650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.737395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.739123 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.746745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9adf0157744463b99e339f6e944c01b', 'request_id': 'req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.752859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1402/2829] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:43:05.754180 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.757507 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9adf0157744463b99e339f6e944c01b', 'request_id': 'req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.763710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9adf0157744463b99e339f6e944c01b', 'request_id': 'req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.764371 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9509ff4-4936-4b20-bfc2-c112f446fda6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.775479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] The token cache is being invalidate because project d9adf0157744463b99e339f6e944c01b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:05.775908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.776625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-70bb8dcd-677c-42bb-a3b5-d70e2c5183d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:05.779170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1428/2830] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:05 2022] DELETE /identity/v3/projects/d9adf0157744463b99e339f6e944c01b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:05.785132 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6675e-35fe-4565-8580-c419aadee7b1 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.785423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6675e-35fe-4565-8580-c419aadee7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.785702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c6675e-35fe-4565-8580-c419aadee7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.805061 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-26c6675e-35fe-4565-8580-c419aadee7b1 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:05.806290 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-26c6675e-35fe-4565-8580-c419aadee7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.831909 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.832598 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.832892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.833163 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.833878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.835482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.846426 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1403/2831] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:43:05.850751 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.852569 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.865393 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afebab-5ec7-4627-ae70-08ebb5852f6e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.870063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.871009 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.871307 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.871590 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] PATH_INFO: `/v3/projects/63bf4503a2cb423e8dbe149d2795d08e/users/a862cdc32b134592bf5e14a4fae85911/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.874848 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1429/2832] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:05.882341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.882630 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.882908 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.889012 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bf4503a2cb423e8dbe149d2795d08e, user_id=a862cdc32b134592bf5e14a4fae85911, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.892452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881806c8-0cd3-48d2-b4bc-a1a407c24e0c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:05.900942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:05.902972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:05.904194 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.908710 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:05.908835 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1404/2833] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:05 2022] PUT /identity/v3/projects/63bf4503a2cb423e8dbe149d2795d08e/users/a862cdc32b134592bf5e14a4fae85911/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:05.920947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.921273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.921525 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.952803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] MFA Rules not processed for user `a862cdc32b134592bf5e14a4fae85911`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:05.954026 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:05.966860 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1430/2834] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:05.971057 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:05.980393 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af04dd9-1561-435b-be8f-c70f0ad58f71 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:05.985062 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:05.985722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:05.986024 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:05.986273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:05.986999 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:43:05.987299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:05.988988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.000988 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1431/2835] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:43:06.005279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.015576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ed752-f839-4ea6-a2a3-0a1762b6cfa3 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.020298 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.020948 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.021232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.021509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.022230 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.024411 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.029313 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1405/2836] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:06.033595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.044580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e31fb74-3bea-4f54-9f37-03bff43ef7f7 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.049655 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.050345 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.050628 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.050905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.051597 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.056518 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1432/2837] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:06.056650 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.057888 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.088577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1406/2838] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:06.090428 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.134884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb866e3c-7adc-4571-8a5e-1cd17447682e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.162155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c4d287-a89d-4ca5-b3f2-4b1878ddd2f9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.167423 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.168222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.168580 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.168942 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.169888 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.171947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.203986 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1407/2839] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:06.205056 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.216267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dad299-5fab-4514-a81c-08ba17ef9fd8 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.221151 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.221845 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.222135 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.222460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.223152 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.228019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.239873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1408/2840] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:43:06.241038 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.241929 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.242694 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.243161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.243450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.244427 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.247258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.254846 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564dc1e-0112-4ba8-a36a-101e7519a7e1 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.260924 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.262454 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.263090 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.263282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] PATH_INFO: `/v3/projects/acf2c3b3d123408685b90ef54fec10ff/users/fd2a05a2bd034099949d9d226f7db10b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.279117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1433/2841] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/users => generated 299 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:43:06.281195 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.288760 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf2c3b3d123408685b90ef54fec10ff, user_id=fd2a05a2bd034099949d9d226f7db10b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.291956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.294051 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514199f-9fb8-4b8e-80ad-205f80bab0c1 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.298879 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.299541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.299824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.300115 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.300872 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.302467 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.309037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:06.314028 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1409/2842] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:06 2022] PUT /identity/v3/projects/acf2c3b3d123408685b90ef54fec10ff/users/fd2a05a2bd034099949d9d226f7db10b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:06.316262 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1434/2843] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:43:06.318366 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.318510 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.329016 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b66247-be8b-4c92-94e8-e4a8b7bad5b2 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.340179 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:06.388055 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c2eae-33aa-4231-9824-949e4c3a8a4f admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.392879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.393557 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.393841 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.394118 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.394833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.396483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.398074 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.398763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.399049 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.399318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.400090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.401802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.406794 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1410/2844] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:43:06.410809 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.415090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1435/2845] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:43:06.416498 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.421282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbf330-a749-4495-abaa-c692d2d802d2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.426265 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35583d-fa73-4360-8735-d46da394915d admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.426400 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.427250 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.427535 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.427816 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] PATH_INFO: `/v3/projects/21d3e63905c0403c9f3c85e1047225e4/users/bda272c6d8a24f32aa1f7a3a972d5b00/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.431102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.431971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.432293 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.432573 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] PATH_INFO: `/v3/projects/acf2c3b3d123408685b90ef54fec10ff/users/fd2a05a2bd034099949d9d226f7db10b/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.435610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf2c3b3d123408685b90ef54fec10ff, user_id=fd2a05a2bd034099949d9d226f7db10b, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.438617 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247ac08-34ca-42a8-9b7a-2f53bd5d66f7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.443526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d3e63905c0403c9f3c85e1047225e4, user_id=bda272c6d8a24f32aa1f7a3a972d5b00, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.446298 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab7fdfa-6f1b-4d1b-ad68-a5cd3188d8e0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.452200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1436/2846] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:06 2022] PUT /identity/v3/projects/acf2c3b3d123408685b90ef54fec10ff/users/fd2a05a2bd034099949d9d226f7db10b/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:06.461661 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1411/2847] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:06 2022] PUT /identity/v3/projects/21d3e63905c0403c9f3c85e1047225e4/users/bda272c6d8a24f32aa1f7a3a972d5b00/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:43:06.465267 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.465557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.465831 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.471982 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.472197 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.472490 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.496341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] MFA Rules not processed for user `fd2a05a2bd034099949d9d226f7db10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:06.497462 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:06.499552 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] MFA Rules not processed for user `bda272c6d8a24f32aa1f7a3a972d5b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:06.500791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:06.569777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1412/2848] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:06.573245 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1437/2849] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:06.581453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.582849 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:06.584854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:06.586227 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:06.644897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e601a7e-014d-457e-a3d2-b9edc6af3206 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.645890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2092bfa-e8c7-4c06-91e0-589309a99724 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:06.648637 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.649273 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.649555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.649645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:06.649832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.650268 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:06.650587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:06.650864 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:06.653139 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:06.661802 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.663554 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.673334 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1413/2850] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:06.677482 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:06.730863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:06.732318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:06.741586 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1438/2851] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:43:08.394864 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:08.408047 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92a75-2dc8-4a2c-bab2-7808db9197bb service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:08.413129 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:08.413857 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:08.414577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:08.414948 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:08.448467 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:08.487972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:08.489475 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:08.500200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1414/2852] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:09.921376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:09.929890 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bd219-63fc-4cf6-899f-f8d1783b9383 service cinder] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:09.933449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:09.934082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:09.934366 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:09.934645 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:09.944821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:09.946256 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f4803-f7f5-4817-b474-93c38ea680c4 service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:09.955674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1439/2853] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:12.735573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:12.737169 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:12.823217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5447d694-f119-4806-80c5-766d95702205 service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:12.828132 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:12.829029 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:12.829316 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:12.829599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] PATH_INFO: `/v3/users/e2b9adbe60b340f4a7e3ef4e55ba5104` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:12.830826 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2b9adbe60b340f4a7e3ef4e55ba5104)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:12.832807 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:12.895789 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2b9adbe60b340f4a7e3ef4e55ba5104'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:12.896630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2b9adbe60b340f4a7e3ef4e55ba5104'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:12.897378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2b9adbe60b340f4a7e3ef4e55ba5104'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:12.906589 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2b9adbe60b340f4a7e3ef4e55ba5104'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:12.923261 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2b9adbe60b340f4a7e3ef4e55ba5104'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:12.937453 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1415/2854] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:12 2022] DELETE /identity/v3/users/e2b9adbe60b340f4a7e3ef4e55ba5104 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:12.941873 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:12.942355 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:12.942486 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:12.960263 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:12.961463 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:12.975120 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:12.976516 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:13.032031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1440/2855] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:13.034459 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:13.038635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:13.040532 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:13.053822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a64d-a5ba-41ea-b401-e57bc126b71b None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:13.054067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082937e-59ba-4fb2-9aba-80465fa095be admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:13.057650 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:13.058061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:13.058306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:13.058590 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:13.058712 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:13.058868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:13.059000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:13.059279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:13.061700 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:13.069106 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:13.070541 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:13.081008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1441/2856] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:43:13.137543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:13.137543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:13.147595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1416/2857] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:43:13.409809 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:13.411602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:13.433254 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba227b0e-c6dd-44c1-8fde-3439ded8ebdb service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:13.439726 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:13.441258 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:13.441538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:13.441821 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] PATH_INFO: `/v3/projects/044e8882fac7469a95707bc6ff31dab4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:13.443576 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=044e8882fac7469a95707bc6ff31dab4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:13.446288 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:13.521951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '044e8882fac7469a95707bc6ff31dab4', 'request_id': 'req-e1d170da-0c3f-419b-9928-b214ebf597e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:13.537283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '044e8882fac7469a95707bc6ff31dab4', 'request_id': 'req-e1d170da-0c3f-419b-9928-b214ebf597e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:13.545376 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '044e8882fac7469a95707bc6ff31dab4', 'request_id': 'req-e1d170da-0c3f-419b-9928-b214ebf597e4'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:13.558450 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] The token cache is being invalidate because project 044e8882fac7469a95707bc6ff31dab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:13.558755 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:13.559555 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:13.561562 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1442/2858] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:13 2022] DELETE /identity/v3/projects/044e8882fac7469a95707bc6ff31dab4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:14.953154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:14.954783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:14.968973 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:14.969572 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b86e58-9774-4102-a126-0eff8bc0b16f service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:14.970491 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:15.003583 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:15.039259 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:15.040154 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.040452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.040734 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] PATH_INFO: `/v3/users/fcf7034654e24046be8a82cb140393bd` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.041944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf7034654e24046be8a82cb140393bd)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:15.044067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:15.062940 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d170da-0c3f-419b-9928-b214ebf597e4 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:15.068520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:15.069405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.069693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.069974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] PATH_INFO: `/v3/users/bda272c6d8a24f32aa1f7a3a972d5b00` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.071341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda272c6d8a24f32aa1f7a3a972d5b00)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:15.073423 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:15.132370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf7034654e24046be8a82cb140393bd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.134912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf7034654e24046be8a82cb140393bd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.136480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf7034654e24046be8a82cb140393bd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.150905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf7034654e24046be8a82cb140393bd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.168662 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda272c6d8a24f32aa1f7a3a972d5b00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.178293 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-d66fc006-8e1b-41ec-a525-1fc73a459887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf7034654e24046be8a82cb140393bd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.184931 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda272c6d8a24f32aa1f7a3a972d5b00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.187066 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda272c6d8a24f32aa1f7a3a972d5b00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.198182 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda272c6d8a24f32aa1f7a3a972d5b00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.203072 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1417/2859] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:14 2022] DELETE /identity/v3/users/fcf7034654e24046be8a82cb140393bd => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:15.205538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.205990 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.206823 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.230279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-ead58306-6102-4085-a7eb-5436ed43b45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda272c6d8a24f32aa1f7a3a972d5b00'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:15.231034 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:15.235834 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:15.249195 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:15.255545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1443/2860] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:14 2022] DELETE /identity/v3/users/bda272c6d8a24f32aa1f7a3a972d5b00 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:15.258565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.259180 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.259646 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.287025 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:15.288564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:15.350632 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1418/2861] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:15.351771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:15.353722 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:15.404281 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1444/2862] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:15.414697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:15.416478 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:15.441654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b7e4b-e463-496d-bb06-5d5786c1b5e0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:15.493368 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:15.519424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6941f8c-b5e9-41d2-be9f-218571722913 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:15.524033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:15.524862 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.525351 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.525846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.537304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:15.539497 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:15.552184 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1445/2863] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:15.556174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:15.556946 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:15.557234 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:15.557517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:15.571200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:15.572776 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:15.585648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1419/2864] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:16.235604 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:16.237100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:16.259763 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545acabe-735d-4b67-950a-901903c8b8f6 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:16.259984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:16.262277 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:16.267517 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:16.268564 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:16.268923 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:16.269277 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] PATH_INFO: `/v3/projects/21d3e63905c0403c9f3c85e1047225e4` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:16.273161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d3e63905c0403c9f3c85e1047225e4)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:16.278142 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:16.303370 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f089461-e1b9-4be0-b20f-0bcde546824e service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:16.313016 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:16.314207 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:16.314604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:16.314865 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] PATH_INFO: `/v3/projects/447e2de61cd84ac3887260319161e817` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:16.316226 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=447e2de61cd84ac3887260319161e817)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:16.319045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:16.349990 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d3e63905c0403c9f3c85e1047225e4', 'request_id': 'req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.366238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d3e63905c0403c9f3c85e1047225e4', 'request_id': 'req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.376947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '447e2de61cd84ac3887260319161e817', 'request_id': 'req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.377769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d3e63905c0403c9f3c85e1047225e4', 'request_id': 'req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.383220 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '447e2de61cd84ac3887260319161e817', 'request_id': 'req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.390844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '447e2de61cd84ac3887260319161e817', 'request_id': 'req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.393716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:16.401683 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] The token cache is being invalidate because project 21d3e63905c0403c9f3c85e1047225e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:16.402020 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.402883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.420141 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1446/2865] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:16 2022] DELETE /identity/v3/projects/21d3e63905c0403c9f3c85e1047225e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:16.421245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] The token cache is being invalidate because project 447e2de61cd84ac3887260319161e817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:16.421245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.421245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.421245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1420/2866] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:16 2022] DELETE /identity/v3/projects/447e2de61cd84ac3887260319161e817 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:16.421652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:16.421652 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:16.518676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1a0f1-3fee-4717-ad36-7b2dc221aa3e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:16.597400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:16.598365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:16.598649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:16.598993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] PATH_INFO: `/v3/users/a862cdc32b134592bf5e14a4fae85911` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:16.600503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a862cdc32b134592bf5e14a4fae85911)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:16.602579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:16.621161 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:16.676364 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a862cdc32b134592bf5e14a4fae85911'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.683606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a862cdc32b134592bf5e14a4fae85911'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.684322 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a862cdc32b134592bf5e14a4fae85911'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.693845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a862cdc32b134592bf5e14a4fae85911'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.712149 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a862cdc32b134592bf5e14a4fae85911'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:16.718781 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1447/2867] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:16 2022] DELETE /identity/v3/users/a862cdc32b134592bf5e14a4fae85911 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:17.138255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:17.139839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:17.152709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2dc93d-9d4a-4b06-9b3c-9d18eb830bb4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:17.219221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:17.220000 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:17.220325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:17.220611 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] PATH_INFO: `/v3/projects/63bf4503a2cb423e8dbe149d2795d08e` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:17.221803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bf4503a2cb423e8dbe149d2795d08e)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:17.223705 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:17.277836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bf4503a2cb423e8dbe149d2795d08e', 'request_id': 'req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.287873 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bf4503a2cb423e8dbe149d2795d08e', 'request_id': 'req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.295957 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bf4503a2cb423e8dbe149d2795d08e', 'request_id': 'req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.310984 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] The token cache is being invalidate because project 63bf4503a2cb423e8dbe149d2795d08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:17.311279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.312005 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.313821 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1421/2868] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:17 2022] DELETE /identity/v3/projects/63bf4503a2cb423e8dbe149d2795d08e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:17.317863 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:17.319301 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:17.331745 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220934f2-eb13-4079-a7c8-108fa25a9749 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:17.449356 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:17.449945 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:17.450794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:17.451090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] PATH_INFO: `/v3/users/fd2a05a2bd034099949d9d226f7db10b` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:17.453386 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2a05a2bd034099949d9d226f7db10b)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:17.455336 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:17.550324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2a05a2bd034099949d9d226f7db10b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.563499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2a05a2bd034099949d9d226f7db10b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.564741 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2a05a2bd034099949d9d226f7db10b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.579128 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2a05a2bd034099949d9d226f7db10b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.597846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-45dd05ce-aa98-4fc6-8cf0-bc6d81d3532e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2a05a2bd034099949d9d226f7db10b'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:17.606214 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1448/2869] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:17 2022] DELETE /identity/v3/users/fd2a05a2bd034099949d9d226f7db10b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:17.946465 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:17.948065 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:17.963524 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b42fa-5d13-4744-8bb3-ae9faf7a4b72 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.044716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.045684 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.046042 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.046409 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] PATH_INFO: `/v3/projects/acf2c3b3d123408685b90ef54fec10ff` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.047746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf2c3b3d123408685b90ef54fec10ff)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.049878 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.091958 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:18.115537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf2c3b3d123408685b90ef54fec10ff', 'request_id': 'req-52b258c0-9ead-484e-9f5a-b631825b814a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:18.124697 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf2c3b3d123408685b90ef54fec10ff', 'request_id': 'req-52b258c0-9ead-484e-9f5a-b631825b814a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:18.132319 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf2c3b3d123408685b90ef54fec10ff', 'request_id': 'req-52b258c0-9ead-484e-9f5a-b631825b814a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:18.147134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] The token cache is being invalidate because project acf2c3b3d123408685b90ef54fec10ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:18.147499 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:18.149149 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-52b258c0-9ead-484e-9f5a-b631825b814a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:18.151480 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1422/2870] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:17 2022] DELETE /identity/v3/projects/acf2c3b3d123408685b90ef54fec10ff => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:18.156749 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929077-7107-4604-8211-7e784923775a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.157221 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929077-7107-4604-8211-7e784923775a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.157506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929077-7107-4604-8211-7e784923775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.178899 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d8929077-7107-4604-8211-7e784923775a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:18.180173 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d8929077-7107-4604-8211-7e784923775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:18.185765 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:18.240766 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1449/2871] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:18.248274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.248832 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.249114 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.267054 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:18.268235 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:18.276905 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1423/2872] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:18.281629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929077-7107-4604-8211-7e784923775a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.291200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929077-7107-4604-8211-7e784923775a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.296508 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.297163 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.297456 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.297737 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.298465 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80136) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:43:18.298777 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.300405 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.315509 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1450/2873] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:43:18.316582 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.325710 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb2c2-e0c6-4919-9dfd-d5cd136bf87f None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.330416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.331067 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.331356 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.331635 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.332378 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.334280 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.362317 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1424/2874] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:43:18 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:43:18.366345 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.376845 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469619b-804c-4ab2-a493-d735215055ce admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.432663 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.433321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.433606 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.433880 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.434628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.436584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.455476 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1451/2875] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:43:18 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:43:18.459709 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.470519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3538fd-5778-4e91-bd9b-d36391135cf9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.475327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.476014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.476340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.476644 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.477369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.478963 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.490119 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1425/2876] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:43:18.493993 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.504134 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfdba6-6e31-45b9-b8cb-5baa60f29aa7 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.508728 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.509371 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.509655 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.509932 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.510648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.512260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744cf353-16b5-4560-96d5-154f13235e02 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.522591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1452/2877] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:43:18.526648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.537206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12cd74-73a8-437f-922e-9cf29ad14edc admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.542338 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.543213 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.543501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.543780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] PATH_INFO: `/v3/projects/12e43b52db1a4e909d7f11368c2b7d97/users/5c11f57e899148da8cafdc6443445a58/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.558204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e43b52db1a4e909d7f11368c2b7d97, user_id=5c11f57e899148da8cafdc6443445a58, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.560966 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.573664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1426/2878] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:43:18 2022] PUT /identity/v3/projects/12e43b52db1a4e909d7f11368c2b7d97/users/5c11f57e899148da8cafdc6443445a58/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:18.579048 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04d91-3923-4574-ba71-c08f327ba1fa None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.579369 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04d91-3923-4574-ba71-c08f327ba1fa None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.579675 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc04d91-3923-4574-ba71-c08f327ba1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.597676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-dfc04d91-3923-4574-ba71-c08f327ba1fa None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:18.598871 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-dfc04d91-3923-4574-ba71-c08f327ba1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:18.655985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1453/2879] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:43:18.665267 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:18.666944 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:18.724681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1100c6-6bae-49c5-abfc-30dc2a68bf4e admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:18.728389 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:18.729032 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:18.729320 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:18.729599 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:18.738500 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:18.739936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:18.749658 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1427/2880] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:43:19.333674 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1841d-c1ac-46b1-bc27-bb0ca69ead15 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:19.333974 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1841d-c1ac-46b1-bc27-bb0ca69ead15 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:19.334306 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1841d-c1ac-46b1-bc27-bb0ca69ead15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:19.360866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-44c1841d-c1ac-46b1-bc27-bb0ca69ead15 None None] MFA Rules not processed for user `5c11f57e899148da8cafdc6443445a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:19.362082 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-44c1841d-c1ac-46b1-bc27-bb0ca69ead15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:19.368800 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:19.434034 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1454/2881] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:19.445200 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:19.446704 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:19.513110 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ae6ef-d5ac-4f5d-b6ee-a22befdf9ef0 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:19.516917 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:19.517703 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:19.518131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:19.518482 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:19.524913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:19.527791 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:19.529481 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:19.540222 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1428/2882] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:20.888878 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81bd1-0c06-41e9-bdb1-4706e301affc None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:20.889339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81bd1-0c06-41e9-bdb1-4706e301affc None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:20.889762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81bd1-0c06-41e9-bdb1-4706e301affc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:20.915413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-84e81bd1-0c06-41e9-bdb1-4706e301affc None None] MFA Rules not processed for user `5c11f57e899148da8cafdc6443445a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:20.915413 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-84e81bd1-0c06-41e9-bdb1-4706e301affc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:20.920047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1455/2883] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:25.407767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:25.409501 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:25.423559 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98ee1e-556a-4c82-a48d-5136d6cb2e4e service cinder] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:25.428216 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:25.429157 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:25.429510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:25.429866 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] PATH_INFO: `/v3/users/e21c4b08ff064b228dd55398d14fed5a` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:25.431147 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21c4b08ff064b228dd55398d14fed5a)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:25.433306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:25.489837 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21c4b08ff064b228dd55398d14fed5a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:25.490505 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21c4b08ff064b228dd55398d14fed5a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:25.491325 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21c4b08ff064b228dd55398d14fed5a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:25.500894 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21c4b08ff064b228dd55398d14fed5a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:25.513241 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21c4b08ff064b228dd55398d14fed5a'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:25.525927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1429/2884] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:43:25 2022] DELETE /identity/v3/users/e21c4b08ff064b228dd55398d14fed5a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:25.530441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:25.530896 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:25.530997 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:25.548551 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:25.550043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:25.610906 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1456/2885] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:25.620715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:25.622740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:25.693511 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3f511-3c22-4fa4-a09b-0864bbdfb7d2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:25.697140 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:25.697783 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:25.698069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:25.698346 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:25.707542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:25.709020 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:25.719931 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1430/2886] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 07:43:25.992698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:25.994164 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:26.007043 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556c711-330e-4e44-bfa3-c8a829a1a286 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:26.011743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:26.013270 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:26.013557 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:26.013833 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] PATH_INFO: `/v3/projects/5edcf34bc63a4954a93eb2b4c72869dd` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:26.015056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5edcf34bc63a4954a93eb2b4c72869dd)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:26.017090 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:26.064661 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5edcf34bc63a4954a93eb2b4c72869dd', 'request_id': 'req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:26.071628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5edcf34bc63a4954a93eb2b4c72869dd', 'request_id': 'req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:26.077589 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5edcf34bc63a4954a93eb2b4c72869dd', 'request_id': 'req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:26.088479 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] The token cache is being invalidate because project 5edcf34bc63a4954a93eb2b4c72869dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:43:26.088769 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:26.089569 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-1a46e5cc-af52-4a3a-8ccf-96a414c1be27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:43:26.091181 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1457/2887] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:43:25 2022] DELETE /identity/v3/projects/5edcf34bc63a4954a93eb2b4c72869dd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:43:33.043263 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:33.045255 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:33.056699 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f7d54-b0c9-4873-a9c0-e288048aec02 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:33.108349 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:33.109224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:33.109498 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:33.109779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:33.111736 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:33.122324 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:33.124301 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:33.136060 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1431/2888] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:34.200902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:34.201404 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:34.201493 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:34.220387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] MFA Rules not processed for user `5c11f57e899148da8cafdc6443445a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:34.221610 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:34.241930 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:34.283473 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1458/2889] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:43:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:34.680010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:34.681542 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:34.687246 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:34.692573 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0227da-809c-45d7-a4ff-49380e9ab5fa service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:34.700340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:34.701004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:34.701448 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:34.701921 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:34.703681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:34.716205 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:34.717649 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885c89-cf13-46b1-b7de-782b98883c30 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:34.726468 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1432/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:43.990135 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:44.000648 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66a2e-977e-463a-a357-a3d0ab9751c6 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:44.004232 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:44.004868 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:44.005155 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:44.005437 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:44.007261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:44.076667 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:44.078643 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:44.089565 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1459/2891] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:43:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:43:44.722810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:54.025279 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa133b-c0cd-4724-8668-a315f84f68c4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:54.025884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa133b-c0cd-4724-8668-a315f84f68c4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:54.025884 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa133b-c0cd-4724-8668-a315f84f68c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:54.043427 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-ecfa133b-c0cd-4724-8668-a315f84f68c4 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:43:54.044642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ecfa133b-c0cd-4724-8668-a315f84f68c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:54.084629 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=48 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:43:54.100518 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1433/2892] 10.222.0.23 () {58 vars in 1009 bytes} [Mon Jun 20 07:43:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:43:54.141347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:43:54.142753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:43:54.212449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e6550-8215-4789-88dd-1d330732ad12 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:43:54.217504 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:43:54.218795 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:43:54.219008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:43:54.219283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:43:54.227208 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:43:54.228811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:43:54.237310 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1460/2893] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:44:04.093540 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=25 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:04.233627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=32 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:08.824972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:08.825476 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:08.825567 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:08.844351 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:08.845842 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:08.853104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1434/2894] 10.222.0.23 () {58 vars in 967 bytes} [Mon Jun 20 07:44:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:08.865671 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:08.876846 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e640edf-f0fd-4f12-a63c-d8a65d796ba0 service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:08.881852 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:08.882485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:08.882817 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:08.883097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:08.891776 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:08.893387 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:08.901513 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1461/2895] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:44:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:44:10.816965 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:10.818412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:10.884956 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bead8e-eca4-44fc-ae3e-f46607e4bca0 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:10.889397 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:10.890815 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:10.891100 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:10.891376 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] PATH_INFO: `/v3/users/5c11f57e899148da8cafdc6443445a58` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:10.892595 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c11f57e899148da8cafdc6443445a58)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:10.894510 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:10.938739 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c11f57e899148da8cafdc6443445a58'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:10.939318 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c11f57e899148da8cafdc6443445a58'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:10.939918 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c11f57e899148da8cafdc6443445a58'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:10.946538 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c11f57e899148da8cafdc6443445a58'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:10.958461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0069fb59-0864-446e-83a0-877e7e4687ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c11f57e899148da8cafdc6443445a58'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:10.970282 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1435/2896] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:44:10 2022] DELETE /identity/v3/users/5c11f57e899148da8cafdc6443445a58 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:11.141753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.152790 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d79fce-7238-4a21-9fd7-72c2c58ba79d service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.212422 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.213219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.213506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.213788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] PATH_INFO: `/v3/projects/12e43b52db1a4e909d7f11368c2b7d97` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.214960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e43b52db1a4e909d7f11368c2b7d97)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.216972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.262332 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e43b52db1a4e909d7f11368c2b7d97', 'request_id': 'req-9a79ccaf-767a-4719-a286-2c3a8d51b727'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:11.269718 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e43b52db1a4e909d7f11368c2b7d97', 'request_id': 'req-9a79ccaf-767a-4719-a286-2c3a8d51b727'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:11.275771 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e43b52db1a4e909d7f11368c2b7d97', 'request_id': 'req-9a79ccaf-767a-4719-a286-2c3a8d51b727'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:11.287733 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] The token cache is being invalidate because project 12e43b52db1a4e909d7f11368c2b7d97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:44:11.288023 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:11.288693 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-9a79ccaf-767a-4719-a286-2c3a8d51b727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:11.290257 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1462/2897] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:44:11 2022] DELETE /identity/v3/projects/12e43b52db1a4e909d7f11368c2b7d97 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:11.295170 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.295581 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.295913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.313777 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:11.315014 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:11.374679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1436/2898] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:11.382231 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.382547 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.382815 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.402102 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:11.403320 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:11.412044 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1463/2899] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:11.416306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.426624 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c4ee4-d55a-4e0f-886a-acc7e3cef2d4 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.431249 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.431892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.432212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.432496 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.433229 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:44:11.433537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.435131 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.443879 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:11.450949 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1437/2900] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 07:44:11.452197 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.464542 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b441da-e931-4d51-bc00-77d74af2d010 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.469698 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.470349 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.470635 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.470912 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.471601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.473533 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.499076 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1464/2901] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:44:11 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:44:11.503336 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.513224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77affd1-4af3-499b-b952-58726c36dff4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.569051 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.569707 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.569998 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.570274 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.570962 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.572897 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.591654 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1438/2902] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:44:11 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:44:11.596299 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.606455 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c4a1b-3dc6-4683-b99e-cf471592a26e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.611681 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.612359 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.612649 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.612927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.613682 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.615260 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.626660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1465/2903] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:44:11.630303 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.639985 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566b22-c048-46a1-ba1b-194baaa4ecd5 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.644577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.645212 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.645495 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.645771 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.646483 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.648063 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4880cd5-1509-4c5f-91cc-463d7d70fcce admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.658294 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1439/2904] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:44:11.662183 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.672602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f0342-d30c-433d-afea-e2a222c5cbd8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.677225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.678087 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.678377 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.678672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] PATH_INFO: `/v3/projects/b8385fa6cdb94069a0ec5397843d16e8/users/536934b7a35d4f51934f53a198201d25/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.692956 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8385fa6cdb94069a0ec5397843d16e8, user_id=536934b7a35d4f51934f53a198201d25, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.695656 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.708895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1466/2905] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:44:11 2022] PUT /identity/v3/projects/b8385fa6cdb94069a0ec5397843d16e8/users/536934b7a35d4f51934f53a198201d25/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 07:44:11.719543 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.719850 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.720168 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.746927 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] MFA Rules not processed for user `536934b7a35d4f51934f53a198201d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:11.748634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:11.813466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1440/2906] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:11.824385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:11.826037 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:11.840485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:11.884408 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e688425-b9d3-4a71-849e-773fed313a50 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:11.887991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '468b120f57fe4807b969c390cbcabfbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:11.888657 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:11.888947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:11.889225 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:11.897883 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:11.899295 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca28f6-0981-4b1a-aa74-1713ce6d44dd service cinder] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:11.908664 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1467/2907] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:44:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:44:14.435642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:14.497755 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3809d5d-ec2e-4b7d-9980-20ba2c97bf5e None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:14.502583 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:14.503434 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:14.503720 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:14.504023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] PATH_INFO: `/v3/users/536934b7a35d4f51934f53a198201d25` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:14.505221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536934b7a35d4f51934f53a198201d25)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:14.507121 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:14.517230 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:14.551716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536934b7a35d4f51934f53a198201d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:14.552390 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536934b7a35d4f51934f53a198201d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:14.552992 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536934b7a35d4f51934f53a198201d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:14.559910 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536934b7a35d4f51934f53a198201d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:14.571577 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536934b7a35d4f51934f53a198201d25'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:14.583466 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1441/2908] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:44:14 2022] DELETE /identity/v3/users/536934b7a35d4f51934f53a198201d25 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:14.588067 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:14.588544 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:14.588696 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:14.606085 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:14.607283 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:14.663960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1468/2909] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:44:14.674085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:14.675460 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:14.734211 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fa81d-d88b-4904-be34-c1ba2a13e7f6 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:14.737847 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:14.738493 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:14.738778 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:14.739061 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:14.748306 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:14.749955 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dde726-7ab8-45e6-8dc5-8598a9ffe2a2 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:14.759300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1442/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:44:15.023304 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:15.024758 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:15.038685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9fedd4-78f9-43b4-9447-39df2d7c61b4 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:15.043545 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:15.044995 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:15.045296 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:15.045757 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] PATH_INFO: `/v3/projects/b8385fa6cdb94069a0ec5397843d16e8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:15.046966 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8385fa6cdb94069a0ec5397843d16e8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:15.049081 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:15.095568 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8385fa6cdb94069a0ec5397843d16e8', 'request_id': 'req-6b026afa-24f0-435e-8205-91a2e6b61dae'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:15.102602 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8385fa6cdb94069a0ec5397843d16e8', 'request_id': 'req-6b026afa-24f0-435e-8205-91a2e6b61dae'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:15.108238 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8385fa6cdb94069a0ec5397843d16e8', 'request_id': 'req-6b026afa-24f0-435e-8205-91a2e6b61dae'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:15.118877 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] The token cache is being invalidate because project b8385fa6cdb94069a0ec5397843d16e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:44:15.119167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:15.119794 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:15.121354 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1469/2911] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:44:15 2022] DELETE /identity/v3/projects/b8385fa6cdb94069a0ec5397843d16e8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:24.755134 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:25.116006 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:25.542248 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:25.542539 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:25.542820 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:25.560868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] MFA Rules not processed for user `c5eecb100c404edba01e417cc4baa10b`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:25.562269 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:25.616586 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1443/2912] 10.222.0.23 () {58 vars in 967 bytes} [Mon Jun 20 07:44:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:25.629112 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:25.630528 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:25.698211 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b026afa-24f0-435e-8205-91a2e6b61dae admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:25.703284 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '601afd44d1f54f939e87511ee463cc4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:25.703876 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:25.704193 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:25.704584 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:25.712884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:25.714503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:25.723251 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1470/2913] 10.222.0.23 () {60 vars in 1384 bytes} [Mon Jun 20 07:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 07:44:32.240975 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:32.244330 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:32.324852 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6b3aa-d5d4-4b8a-ae04-7524fc807810 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:32.329514 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:32.330380 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:32.330664 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:32.331104 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] PATH_INFO: `/v3/users/eda8b1053f95458abfd8ea79227a0dcf` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:32.332341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eda8b1053f95458abfd8ea79227a0dcf)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:32.334283 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:32.387936 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eda8b1053f95458abfd8ea79227a0dcf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.388634 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eda8b1053f95458abfd8ea79227a0dcf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.389245 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eda8b1053f95458abfd8ea79227a0dcf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.397049 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eda8b1053f95458abfd8ea79227a0dcf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.409754 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eda8b1053f95458abfd8ea79227a0dcf'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.421593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1444/2914] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:44:32 2022] DELETE /identity/v3/users/eda8b1053f95458abfd8ea79227a0dcf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:32.595400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:32.607951 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ad7a7-4fa8-4d3c-9316-120b85dc2f0a service swift] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:32.665488 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:32.666273 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:32.666559 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:32.666843 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] PATH_INFO: `/v3/projects/0340346ca5a4424fa8a5374af75990da` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:32.668056 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0340346ca5a4424fa8a5374af75990da)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:32.670027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:32.717113 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0340346ca5a4424fa8a5374af75990da', 'request_id': 'req-5b3660e5-eb31-49ec-a045-2f871c902c4a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.724811 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0340346ca5a4424fa8a5374af75990da', 'request_id': 'req-5b3660e5-eb31-49ec-a045-2f871c902c4a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.730219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0340346ca5a4424fa8a5374af75990da', 'request_id': 'req-5b3660e5-eb31-49ec-a045-2f871c902c4a'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.741098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] The token cache is being invalidate because project 0340346ca5a4424fa8a5374af75990da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:44:32.741390 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.742027 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.743567 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1471/2915] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:44:32 2022] DELETE /identity/v3/projects/0340346ca5a4424fa8a5374af75990da => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:32.747633 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:32.749369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:32.762436 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b5406-f39f-4886-a23d-eb43d68ea322 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:32.783369 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:32.819973 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:32.820859 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:32.821143 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:32.821421 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] PATH_INFO: `/v3/users/a091e587a9c547cca66b526425afc2c8` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:32.822593 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a091e587a9c547cca66b526425afc2c8)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:32.824562 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:32.869004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a091e587a9c547cca66b526425afc2c8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.869645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a091e587a9c547cca66b526425afc2c8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.870266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a091e587a9c547cca66b526425afc2c8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.877770 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a091e587a9c547cca66b526425afc2c8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.889922 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-0419b7ba-18cf-4e3f-98e8-528df3b0a9ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a091e587a9c547cca66b526425afc2c8'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:32.904432 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1445/2916] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:44:32 2022] DELETE /identity/v3/users/a091e587a9c547cca66b526425afc2c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:33.072321 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.073916 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:33.089538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3660e5-eb31-49ec-a045-2f871c902c4a admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.106117 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:33.150287 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.151147 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.151432 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.151711 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] PATH_INFO: `/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.152914 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d82c5d67dc34d7ba1c7b6c744e0d7a8)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.154807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.200627 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d82c5d67dc34d7ba1c7b6c744e0d7a8', 'request_id': 'req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:33.208495 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d82c5d67dc34d7ba1c7b6c744e0d7a8', 'request_id': 'req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:33.218280 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d82c5d67dc34d7ba1c7b6c744e0d7a8', 'request_id': 'req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:33.229241 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] The token cache is being invalidate because project 8d82c5d67dc34d7ba1c7b6c744e0d7a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80136) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:44:33.229534 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:33.230167 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-8dd7ca73-5c6d-4ce3-baa4-8e1ea8c2e83e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:44:33.231695 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1472/2917] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:44:33 2022] DELETE /identity/v3/projects/8d82c5d67dc34d7ba1c7b6c744e0d7a8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:33.236913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.237208 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.237492 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.255780 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:33.257018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:33.315023 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1446/2918] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:33.323188 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.323485 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.323774 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.342909 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:33.344130 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:33.352070 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1473/2919] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:33.355981 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.366044 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e06de2-50dc-4f69-badd-189a3e009221 None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.370716 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.371359 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.371638 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.371913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.372673 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80137) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 07:44:33.373240 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.374839 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.389094 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1447/2920] 10.222.0.23 () {58 vars in 1152 bytes} [Mon Jun 20 07:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 07:44:33.393489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.403503 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a24fc-f2ec-4378-8277-7acb08f8988a None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.408170 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.408822 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.409107 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.409384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.410084 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.411986 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.437438 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1474/2921] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:44:33 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:44:33.442265 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.452340 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515497-e057-4fda-94bc-4f364eafb1c9 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.480740 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:33.511615 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.512305 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.512592 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.512868 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.513564 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.515452 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.535519 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1448/2922] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:44:33 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:44:33.539587 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.550305 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ab04a-ad28-494c-bd0a-07b431995675 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.554927 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.555579 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.555866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.556179 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.556935 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.558536 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.570337 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1475/2923] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:44:33.574506 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.584682 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249899df-3251-44cb-bba3-240ffd776fb4 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.589266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.589913 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.590204 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.590488 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.591206 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.592840 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ad1db-907e-4f9f-8926-de83c6ed9e5c admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.602636 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1449/2924] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 07:44:33.606608 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.616928 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3e6e0-4c93-4eb3-8b9a-271bcd23b45b admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.621506 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.622397 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.622676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.623210 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] PATH_INFO: `/v3/projects/c5446ee5342d42a4a39476d540fb03d0/users/8a6bf639c83d4d0ea6f908f3e1f3772c/roles/b1643f7fcdff4280b6a98fb292c0e2cb` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.639094 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5446ee5342d42a4a39476d540fb03d0, user_id=8a6bf639c83d4d0ea6f908f3e1f3772c, role_id=b1643f7fcdff4280b6a98fb292c0e2cb)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.641824 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.654624 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1476/2925] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:44:33 2022] PUT /identity/v3/projects/c5446ee5342d42a4a39476d540fb03d0/users/8a6bf639c83d4d0ea6f908f3e1f3772c/roles/b1643f7fcdff4280b6a98fb292c0e2cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:33.659680 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57f06b-73ab-412c-9333-c73336aeaae7 None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.659976 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57f06b-73ab-412c-9333-c73336aeaae7 None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.660292 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57f06b-73ab-412c-9333-c73336aeaae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.678053 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-6b57f06b-73ab-412c-9333-c73336aeaae7 None None] MFA Rules not processed for user `1ecb0c12822b4be89d3c7e17c8adfe7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:33.679231 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6b57f06b-73ab-412c-9333-c73336aeaae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:33.736947 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1450/2926] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:33.746443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:33.748499 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:33.800457 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:33.806553 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69023594-09db-46b8-b8ad-4f2d55d759c8 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:33.810884 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:33.811521 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:33.811807 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:33.812086 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:33.821019 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:33.822441 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:33.831854 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1477/2927] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:44:37.753327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:37.753811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:37.753811 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:37.784227 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.auth.core [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] MFA Rules not processed for user `8a6bf639c83d4d0ea6f908f3e1f3772c`. Rule list: `[]` (Enabled: `True`). {{(pid=80137) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:37.785555 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:37.860614 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1451/2928] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 07:44:37.864978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:37.875091 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431f64-068d-4016-bc6f-8074a3ade7c1 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:37.880294 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:37.880972 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:37.881353 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:37.881639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:37.882350 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:37.884279 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:37.914047 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1478/2929] 10.222.0.23 () {60 vars in 1152 bytes} [Mon Jun 20 07:44:37 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 07:44:37.916372 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:37.927952 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3022c3-56e3-4cf7-aa68-8418d3eafa8b None None] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:37.967869 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:37.989289 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:37.989970 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:37.990247 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:37.990520 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:37.991215 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:37.993167 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:38.015883 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1452/2930] 10.222.0.23 () {60 vars in 1143 bytes} [Mon Jun 20 07:44:37 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 07:44:38.019836 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:38.030827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a556b35-782d-487b-b578-bb4e5c291132 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:38.035697 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:38.036399 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:38.036687 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:38.036971 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:38.037727 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:38.039339 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:38.051402 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1479/2931] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:44:38.055648 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:38.066537 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af63e63-9f82-4802-8a30-2152adf189b2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:38.071529 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:38.072442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:38.072730 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:38.073006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] PATH_INFO: `/v3/projects/9775230543b14e92bedffdf0923244b7/users/127526ccaa0d43bf8dad5afab6b8c533/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:38.088341 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9775230543b14e92bedffdf0923244b7, user_id=127526ccaa0d43bf8dad5afab6b8c533, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:38.091115 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:38.104979 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1453/2932] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:44:38 2022] PUT /identity/v3/projects/9775230543b14e92bedffdf0923244b7/users/127526ccaa0d43bf8dad5afab6b8c533/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:38.108947 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:38.119524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6cdb2-0481-4752-80b4-bd08df3d6263 admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:38.196406 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:38.197083 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:38.197365 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:38.197672 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:38.198400 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:38.199991 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b2bb6-6b40-4587-afce-3f28fef1ce6a admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:38.215264 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1480/2933] 10.222.0.23 () {58 vars in 1121 bytes} [Mon Jun 20 07:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 07:44:38.219566 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:38.230040 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425394e-0ac2-43da-a378-9a48cc593248 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:38.235651 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:38.236608 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:38.236906 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:38.237155 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] PATH_INFO: `/v3/projects/9775230543b14e92bedffdf0923244b7/users/127526ccaa0d43bf8dad5afab6b8c533/roles/53acd1fbd8c545949ec0b81266c9d3ad` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:38.239174 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9775230543b14e92bedffdf0923244b7, user_id=127526ccaa0d43bf8dad5afab6b8c533, role_id=53acd1fbd8c545949ec0b81266c9d3ad)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:38.241886 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:38.254343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1454/2934] 10.222.0.23 () {60 vars in 1482 bytes} [Mon Jun 20 07:44:38 2022] PUT /identity/v3/projects/9775230543b14e92bedffdf0923244b7/users/127526ccaa0d43bf8dad5afab6b8c533/roles/53acd1fbd8c545949ec0b81266c9d3ad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:44:43.042026 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5332d3-5017-464a-9917-a37ea873c39d None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:43.042658 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5332d3-5017-464a-9917-a37ea873c39d None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:43.043219 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5332d3-5017-464a-9917-a37ea873c39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:43.076097 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-3f5332d3-5017-464a-9917-a37ea873c39d None None] MFA Rules not processed for user `127526ccaa0d43bf8dad5afab6b8c533`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:43.077362 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-3f5332d3-5017-464a-9917-a37ea873c39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:43.183554 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1481/2935] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:43.200844 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:44:43.203972 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:43.290102 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a6bb4-343c-4284-a01d-b37830c1abb0 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:44:43.297224 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:44:43.297930 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:43.298365 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:43.298647 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:43.365790 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:44:43.367413 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:44:43.376779 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1455/2936] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:44:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:44:44.358223 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] REQUEST_METHOD: `POST` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:44:44.358644 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:44:44.358799 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:44:44.383762 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.auth.core [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] MFA Rules not processed for user `8a6bf639c83d4d0ea6f908f3e1f3772c`. Rule list: `[]` (Enabled: `True`). {{(pid=80136) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 07:44:44.385122 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:44:44.393849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1482/2937] 10.222.0.23 () {58 vars in 957 bytes} [Mon Jun 20 07:44:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 07:44:53.372344 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:44:54.374060 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:45:14.664978 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:45:14.666398 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:45:14.731069 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b57817b-881a-4aac-8b18-83528b39e95d service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:45:14.734765 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:45:14.735442 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:45:14.735731 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:45:14.736010 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:45:14.745142 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:45:14.746693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:45:14.759954 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1456/2938] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:45:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:45:24.755558 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=26 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:45:39.434261 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:45:39.435688 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:45:39.509138 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bac9e6-cfd6-4011-a5b8-8d865eca4537 None None] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:45:39.514318 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:45:39.514960 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:45:39.515244 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:45:39.515520 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:45:39.517328 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:45:39.528952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:45:39.530515 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:45:39.540289 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1483/2939] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:45:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:45:49.536080 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=32 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:47:23.707679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:47:23.716642 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00dfd6c-044b-465a-b714-251db2999366 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:47:23.720933 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:47:23.721645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:47:23.722004 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:47:23.722375 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:47:23.724275 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:47:23.736159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:47:23.737604 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:47:23.745926 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1457/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:47:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:47:23.833108 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:47:23.842601 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2f87-f434-43d0-8b8e-87d8b2954775 service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:47:23.846708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:47:23.847341 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:47:23.847625 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:47:23.847902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:47:23.849690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:47:23.863275 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:47:23.864722 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:47:23.875702 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1484/2941] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:47:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:47:33.742159 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=6 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:47:33.870653 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=6 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:48:40.171350 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:40.180146 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8c287-57ab-4e13-8105-13b13256f40b service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:40.183764 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:40.184429 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:40.184715 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:40.185006 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:40.186746 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:40.196803 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:40.198221 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:40.206892 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1458/2942] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:46.096524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:46.097952 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:46.114708 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a16c3-6468-496d-afde-766d51cc3b01 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:46.120677 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:46.122032 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:46.122449 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:46.122738 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:46.124699 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:46.138095 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:46.139685 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:46.152639 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1485/2943] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:46.282019 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:46.290880 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769e2a9b-3f74-43c5-b287-d52e1141e307 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:46.294507 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:46.295141 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:46.295424 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:46.295701 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:46.297526 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:46.311645 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:46.313137 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:46.324333 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1459/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:47.501098 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:47.510538 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e172ec6-4115-425b-976a-f28c00c2bbbf service nova] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:47.514033 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '014dfb1baeb24fa7ab38923a6962db4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:47.514660 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:47.514944 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:47.515218 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:47.516978 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:47.530489 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:47.532385 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:47.543172 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1486/2945] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:48.157574 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:48.170885 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c6117c-289a-4771-8b31-c5f82265c8b4 service neutron] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:48.177412 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e19cf623e22c485fb1bb8f0ada7114b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:48.178085 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:48.178374 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:48.178653 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:48.180517 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:48.193714 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:48.195304 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:48.207407 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1460/2946] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:52.977717 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:52.990744 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e14ce-e3e1-4d44-868e-bbc080307999 service neutron] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:52.997093 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:52.997985 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:52.998274 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:52.998577 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:53.001690 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:53.014556 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:53.016802 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:53.027788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1487/2947] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:56.101239 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:56.120045 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784d756-6cd0-436a-89e5-422ff7088fed service nova] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:56.126098 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:56.126810 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] REQUEST_METHOD: `GET` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:56.127097 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:56.127387 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:56.131681 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:56.145018 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:56.146721 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:56.158836 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1461/2948] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:48:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:48:56.190753 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:56.253796 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ded4c-35cf-4ca8-b65c-e00c3cd7c13c service placement] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:56.258849 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:56.259743 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:56.260031 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:56.260342 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] PATH_INFO: `/v3/users/8a6bf639c83d4d0ea6f908f3e1f3772c` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:56.261524 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6bf639c83d4d0ea6f908f3e1f3772c)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:56.263443 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:56.308953 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6bf639c83d4d0ea6f908f3e1f3772c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.315866 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6bf639c83d4d0ea6f908f3e1f3772c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.316612 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6bf639c83d4d0ea6f908f3e1f3772c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.323895 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6bf639c83d4d0ea6f908f3e1f3772c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.336174 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6bf639c83d4d0ea6f908f3e1f3772c'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.342490 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1488/2949] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:48:56 2022] DELETE /identity/v3/users/8a6bf639c83d4d0ea6f908f3e1f3772c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:48:56.518327 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:56.530395 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376498-d761-4334-8f8d-2ebde51590d7 service placement] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:56.595812 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:56.596630 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:56.596912 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:56.597187 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] PATH_INFO: `/v3/projects/c5446ee5342d42a4a39476d540fb03d0` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:56.598343 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5446ee5342d42a4a39476d540fb03d0)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:56.600266 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:56.645300 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5446ee5342d42a4a39476d540fb03d0', 'request_id': 'req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.650693 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5446ee5342d42a4a39476d540fb03d0', 'request_id': 'req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.656679 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5446ee5342d42a4a39476d540fb03d0', 'request_id': 'req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.669177 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] The token cache is being invalidate because project c5446ee5342d42a4a39476d540fb03d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:48:56.669461 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.670087 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.671609 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1462/2950] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:48:56 2022] DELETE /identity/v3/projects/c5446ee5342d42a4a39476d540fb03d0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:48:56.675539 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:56.677017 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:56.690424 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730740d-f250-4540-baa8-5d7304024aad admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:56.730591 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:48:56.752233 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:56.753100 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:56.753384 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:56.753659 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] PATH_INFO: `/v3/users/127526ccaa0d43bf8dad5afab6b8c533` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:56.754837 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] RBAC: Authorizing `identity:delete_user(user_id=127526ccaa0d43bf8dad5afab6b8c533)` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:56.756788 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:56.802209 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127526ccaa0d43bf8dad5afab6b8c533'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.809144 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127526ccaa0d43bf8dad5afab6b8c533'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.809827 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127526ccaa0d43bf8dad5afab6b8c533'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.816942 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127526ccaa0d43bf8dad5afab6b8c533'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.828472 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.notifications [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127526ccaa0d43bf8dad5afab6b8c533'} {{(pid=80136) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:56.834466 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1489/2951] 10.222.0.23 () {58 vars in 1223 bytes} [Mon Jun 20 07:48:56 2022] DELETE /identity/v3/users/127526ccaa0d43bf8dad5afab6b8c533 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:48:57.001202 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Authenticating user token {{(pid=80137) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:48:57.002660 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.fernet_utils [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80137) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:48:57.016394 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8757a0-fbc9-4d50-be25-fef9ea9b72e2 admin admin] Validating token access rules against request {{(pid=80137) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:48:57.050683 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:48:57.076522 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ecb0c12822b4be89d3c7e17c8adfe7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '649b350e7f364263b37fbd34d0f07b2b', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80137) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:48:57.076937 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:48:57.077217 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:48:57.077494 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] PATH_INFO: `/v3/projects/9775230543b14e92bedffdf0923244b7` {{(pid=80137) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:48:57.078694 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9775230543b14e92bedffdf0923244b7)` {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:48:57.080618 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] RBAC: Authorization granted {{(pid=80137) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:48:57.126455 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9775230543b14e92bedffdf0923244b7', 'request_id': 'req-a8580b37-4353-4476-b4a7-86c1f9425bfd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:57.131833 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9775230543b14e92bedffdf0923244b7', 'request_id': 'req-a8580b37-4353-4476-b4a7-86c1f9425bfd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:57.137786 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9775230543b14e92bedffdf0923244b7', 'request_id': 'req-a8580b37-4353-4476-b4a7-86c1f9425bfd'} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:57.150784 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] The token cache is being invalidate because project 9775230543b14e92bedffdf0923244b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80137) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 07:48:57.151077 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:57.151767 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG keystone.notifications [None req-a8580b37-4353-4476-b4a7-86c1f9425bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80137) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 07:48:57.153355 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: [pid: 80137|app: 0|req: 1463/2952] 10.222.0.23 () {58 vars in 1232 bytes} [Mon Jun 20 07:48:56 2022] DELETE /identity/v3/projects/9775230543b14e92bedffdf0923244b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 07:49:06.830902 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:49:07.147416 dv-cinder-778193-28-iscsi devstack@keystone.service[80137]: DEBUG dbcounter [-] [80137] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80137) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 07:50:46.450723 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Authenticating user token {{(pid=80136) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 07:50:46.452133 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:50:46.549324 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e2eec-f415-4bfc-9f49-6c25b6a2021e admin admin] Validating token access rules against request {{(pid=80136) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 07:50:46.604676 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2aaa83fc8b334b07b67a2ff908a4a105', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd1fe725ff03146febe44bdfa6ff51a78', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80136) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 07:50:46.605347 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] REQUEST_METHOD: `GET` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 07:50:46.605628 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 07:50:46.605907 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80136) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 07:50:46.608200 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.fernet_utils [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80136) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 07:50:46.619008 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 07:50:46.620851 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718f875-463d-46b5-9877-90b73c1ae5c9 service placement] RBAC: Authorization granted {{(pid=80136) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 07:50:46.631013 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: [pid: 80136|app: 0|req: 1490/2953] 10.222.0.23 () {60 vars in 1365 bytes} [Mon Jun 20 07:50:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 07:50:56.626832 dv-cinder-778193-28-iscsi devstack@keystone.service[80136]: DEBUG dbcounter [-] [80136] Writing DB stats keystone:SELECT=71 {{(pid=80136) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}